Lyrics text analysis

Happy new year!

Short version: I tried to plot the word frequency used in lyrics. It looks like this:

This slideshow requires JavaScript.

Just like in literature, the distribution is quite exponential.

You can also find the raw data and the length distribution in the slide show 🙂

Continue reading


Experiencing the cabal hell with Fcomp

This is an effort to try to run the FComp Haskell package described here:

So, with the help from an awesome colleague, the story goes like:

It complained when I did cable install -> delete all sorts of upper cap of dependencies -> can’t install instant-generics 0.6, only instant-generics 0.1 could be successfully installed -> delete instant-generics 0.1, which was not straight forward… one had to find all files on the system and unregister it from ghc-pkg -> leave the mess and try cabal sandbox and install fcomp from scratch -> found out there’s another problem about template haskell -> delete and change all sorts dependency restrictions again, even tried to go from larger to smaller number… used instant-generics 0.4 for example -> then a problem come from uuparsinglib -> then from haskore splitbase -> then from special-functors, and we found out it’s because this package hasn’t been updated for a very long time, maybe got abandoned by the author 😦 and the incompatibility comes from something in Control/Monad/Instances.hs

It’s been the quite a journey and we decided to let it go for now…

Something I learnt from this:

It’s good to save the terminal history as a txt file and look back on it later. Kind of like a cool journal 😀

The importance of Vim and command line is paramount. This is what I’ll be doing for a while now:


The difficult really lies in the intertwined complexity amongst the cabal files. My poor memory is just not holding up.

Why there’s no self-container (with all the proper dependencies versions) to just make everything reproducible at anytime? It might be big but shouldn’t be a problem with current storage ability?


Running harmtrace

To get into the functional programming side, I’m learning more and more about a functional music package in Haskell: harmtrace. It can parse a chord sequence (what I’m using it for), and do much more music analysis in a clean and functional way.

Thanks to the help of the authors of this package, I was finally able to run it using the binary. (Still not able to build it though, because of all the version issues with ghc..)

Some specifics are given here:

Here’s the screen shot of running it in terminal:


The output is a php syntax tree like this:


To visualise it, one can use this website:

And the visualisation looks like this:



In the future, more on the analysis of these trees…