Pattern Recognition Course in Delft

Almost a month ago, I went to this ASCI course to fulfil my “study plan”. It wasn’t very pleasant to get up everyday at 6:30am and take the train to Rotterdam and then bus to Delft, but it was better with my office mates 😛

The course was a good overview of the machine learning and pattern recognition materials I learnt before. Some of them I only learnt on my own and had some loopholes, so it was good to have them solid in this course.  There are new theorems, new concepts and new applications, too, like the impossible theorem for clustering. I now know why some people hate clustering…

One unfortunately thing is that the main package used in the course is in Matlab. I admit it can still be sometimes useful though. I also took the opportunity and learnt Weka. Can be handy, too. The slides and exercises of the course can be found here: http://www.prcourse.net/about/

The campus of TUDelft was quite nice. A little resemblance to Utrecht but seems to be bigger. Had a lunch outside Aula on the grass!

Language learning phase one

I went to the North American Polyglot Symposium in June 2016. My passion for learning languages got rekindled there (it was never completely gone!)

I’ve been listen to many good podcasts about language learning since(I can’t believe I didn’t even know they exist!):

I will teach you a language: http://www.iwillteachyoualanguage.com/category/podcast/

Actual fluency: http://actualfluency.com/actual-fluency-language-learning-podcast/

Sometimes the topic is a little repetitive, but everytime I think about this topic, there’s something new, especially if I’ve learnt a new language.

There are more good resources I’ve been using:

Multilingual DVDs: I have loads of them at home now (thought of using them for research at one point!). It’s very easy to get in Europe. There’re always subbed/dubbed in 3+ languages for one movie/anime/TV show. Now Netflix Originals have this feature, too. Learning with fun!

Various youtube channels like (영국남자 Korean Englishman, Golden MoustacheEasy Languages, etc.) and a Chinese website that has proper lessons: http://www.wanmen.org/#/

The real thing is always practice with native speakers, or just ask them questions. I’m lucky to have many friends who want to like to skype with me. Thank you guys so much! It’s also good to have a teacher who has more experience in teaching the language, and I do pay a little for that. I’m gonna shamely paste my invitation code here (it’s a really nice system though!):

https://www.italki.com/i/CDDc6C

Thank-you-word-cloud

More personal experiences:

The most fun I get from language learning is to look at different representations of the same object and new strutures of new concepts.

I don’t know why but it’s also very automatic to me to repeat after people, especially when it’s a new language and it’s something worth remembering. Imitation, down to every detail, is fun. Strecht the perception!

And plus that I like to talk to myself using different languages: different medolies associate with the same emotion, more ways to express, etc.

Finally, it’s just much more reassuring when you understand more, right?

Commutativity and Associativity

I remember learning associativity and commutativity when I was in elementary school.The formula for the associativity is (a * b) * c =  a * (b * c) and for the commutativity is a * b = b * a , but they look somewhat familiar and confusing to me back then

I think it’d be fun just to draw an illustration:
IMG_2189.PNG
The key is to think of a concrete operation * (the weird box shape in the illustration)

Think of it as a meat grinder or a blackbox or an actual program. Anyway, it takes two input and gives an output.

So now it becomes clear that the associativity is about how to combine the two operations and the commutativity is about the property of the operation itself.

However, there is a connection between them: the associativity is switching the order of which inputs are getting the operation performed first. It’s commutativity “in time”!

If you still don’t get it, here’re more details: Since commutativity is basically just that, when the input order is changed, something preserve, then we have that associativity is actually also a kind of commutativity: it doesn’t matter whether you mix a and b first or b and c first. On the other hand, like we mentioned before, commutativity is more like a switching back and forth in space when preserving some properties.

Anyway, here’s a fun thing. Knowing that in maths commutativity seems to be less common than associativity, I searched for a real life example of being commutative but not associative. Then from the mighty stackoverflow, there is a surprisingly easy one: giving birth! And then it occurs to me that when there’s probability involved, these properties break down…

Going even further, in the category theory, the input becomes associative function (operation). People then explicitly call the diagram commutative!

Learning functional programming and category theory: Monad, Monoid, Complexity

Here’s a video that really helped me to understand all the monad and monoid in functional programming. Also I got a new intuition for category theory as well.

It’s not a perfect video. For me, because I’m a beginner, the tempo of the explaination was sometimes too fast. There are places where it’s very slow as well. But it’s this kind of video that you can’t help but do the pause-rewind-google  digging. Very nice for beginners!

Here are some notes I took when watching the video and from searching for relevant explanation:
Monoid: identity + associativity
It’s about how to combine function, how to build up complexity, take complexity (side effect) into control

Type constructor = extra data

Monad is just composition of functions which return a different type
Bind is the tricky part in Monads (because it lacks symmetry)
Selectmany or flatmap is an example of bind

A bit of history:
Bottom up camp: turning machine, start from the hardware to the abstract
Top down camp: lambda calculus, start from the abstract to implementation

Python this week: for loops, scripting, etc.

I know I shouldn’t write for loops anymore since I’ll be doing functional programming. But it’s still faster for me to do stuff imperatively. Hopefully the transition to FP will come smoothly and soon! (Maybe a post on the learning process. I’m having lots of fun!)

For now, here’s the monstrous 3 level for loop I wrote to do the computation (comparison actually) in this post and this post. There’s actually another one hidden and not shown here because it happens way in the beginning with variable i indicating which music piece it is. But even without any background, I think what I’m trying to say is the way to arrange the order of for loops and how to make hierarchy out of this. It’s pretty straight forward but just need some thinking:

screenshot-from-2017-02-23-23-51-38

When there are lots of for loops, there is a hierarchy of some kind. The way to encode every hierarchy (maximum hierachicality maybe?) in python is to put an empty list before every for loop, and collect the output after the computation in the for loop (I didn’t collect them all because one level of hierachy didn’t matter to me, and I used both list and dict in this one). The order of the for loop (over which variable we should iterate first) is the tricky part. One easy way to start is to think which level of the hierarchy we would like to display first, and put that in the outermost loop. 

About scripting, I think it’s again something which I would frown upon myself in the future. But for the speed I need and my limited programming knowledge, I do use lots of scripts. It’s my way to speed things up with losing the control and reproducibility. It’s very easy to create scripts and keep the complexity of each script under control when the project is small. Naming is also important here: I’m really bad at it, so only when the project is small, I can handle it; otherwise I lose track that which script is associate with which functionality. Good documentation can help with this. 

Well, modularity and compositional complexity always go hand in hand. But the latter is probably more interesting. 

screenshot-from-2017-02-24-00-00-33

Learning graphical models and problog

This is yet another post long overdue….

Along with some other purposes, to get a better foundation in Artificially Intelligence, I revised graphical models and tried out an interesting probabilistic logic programming language “problog”.

(A good starting place: https://dtai.cs.kuleuven.be/problog/tutorial/basic/02_bayes.html)

I’ve not completely understood it yet, but it seems the language is helping in calculating lots of interactions of events according to probability theory and graphical models, etc. It’s been always a very cool idea to me to create automised calculation and machine learning algorithms.

In addition, I would highly recommend these talks by Christopher Bishop, it goes every well with his book Chapter 8:

 

I wish I had watched this before I got to Rochester. I might have understood David Temperley’s model better. But it’s never tool late. There are lots of potential applications of these models in music information research. And I think that might be one crucial aspect that I was lacking of. It’s funny how and what you think and believe can change so drastically after learning a new powerful subject like this: for me, now it’s even strange to not consider an element of probability in the frameworks and algorithms.