Lyrics text analysis

Happy new year!

Short version: I tried to plot the word frequency used in lyrics. It looks like this:

This slideshow requires JavaScript.

Just like in literature, the distribution is quite exponential.

You can also find the raw data and the length distribution in the slide show 🙂

Continue reading

Advertisements

Experiencing the cabal hell with Fcomp

This is an effort to try to run the FComp Haskell package described here: http://dreixel.net/research/pdf/fghm.pdf

So, with the help from an awesome colleague, the story goes like:

It complained when I did cable install -> delete all sorts of upper cap of dependencies -> can’t install instant-generics 0.6, only instant-generics 0.1 could be successfully installed -> delete instant-generics 0.1, which was not straight forward… one had to find all files on the system and unregister it from ghc-pkg -> leave the mess and try cabal sandbox and install fcomp from scratch -> found out there’s another problem about template haskell -> delete and change all sorts dependency restrictions again, even tried to go from larger to smaller number… used instant-generics 0.4 for example -> then a problem come from uuparsinglib -> then from haskore splitbase -> then from special-functors, and we found out it’s because this package hasn’t been updated for a very long time, maybe got abandoned by the author 😦 and the incompatibility comes from something in Control/Monad/Instances.hs

It’s been the quite a journey and we decided to let it go for now…

Something I learnt from this:

It’s good to save the terminal history as a txt file and look back on it later. Kind of like a cool journal 😀

The importance of Vim and command line is paramount. This is what I’ll be doing for a while now:

851796BF-2BCD-4500-9464-217BD48778E4.png

The difficult really lies in the intertwined complexity amongst the cabal files. My poor memory is just not holding up.

Why there’s no self-container (with all the proper dependencies versions) to just make everything reproducible at anytime? It might be big but shouldn’t be a problem with current storage ability?

 

Running harmtrace

To get into the functional programming side, I’m learning more and more about a functional music package in Haskell: harmtrace. It can parse a chord sequence (what I’m using it for), and do much more music analysis in a clean and functional way.

Thanks to the help of the authors of this package, I was finally able to run it using the binary. (Still not able to build it though, because of all the version issues with ghc..)

Some specifics are given here:

http://foswiki.cs.uu.nl/foswiki/GenericProgramming/HarmTrace

Here’s the screen shot of running it in terminal:

Capture.PNG

The output is a php syntax tree like this:

[Piece[PT[T_3par_1[IIIm_0[E:min]]]][PD[D_1_1[S_3_1[IV_0[F:maj7]]][D_1_1[S_1par_1[IIm_0[D:min]]][D_2_1[V7[Inserted]]]]]][PT[T_1_1[I_0[C:maj]]]][PD[D_3_1[V_0[G:maj7][G:maj]]]][PT[T_1_1[I_0[C:maj]]]]]

To visualise it, one can use this website:

http://ironcreek.net/phpsyntaxtree/

And the visualisation looks like this:

 

 

In the future, more on the analysis of these trees…