More Go Playing
I've been getting more into building compiled apps thanks to how easy I'm finding the Go programming language and the Fyne library for cross-platform GUI development. For a while I've been frustrated with the state of the art for free Apache log analysers. There's the old faithful AWStats and Webalizer... but I'm finding their layout and information a bit... old. I'd like something new but I'm a hobbyist and I'm not going to pay for Splunk or similar just to get fast log searching. So why not build my own?1
With a week away with the family in the middle, I've spent some time now playing with building a Go+Fyne+MySQL log file visualiser. So far I've got the following breakdown:
- Digest: Parse through log files (currently in Apache log format) and process them into a standard struct layout
- Decypher: Figure out what the log files mean for data recording and splinter them out into summary tables, perform IP->Country mapping, etc.
- Display: Visualise end point.
3D visualising! Marketable.
I've spent a non-insignificant amount of time playing with SVG world maps to create a visualisation of the top, near top, and also-ran of countries that visit my site. Wikipedia has a few good worldmaps with CC0 licensing2, and there was a great one with country labels in SVG format. Scalable! Unfortunately, it was also processor heavy in Fyne so I ended up grabbing a simpler CC0 map I could translate into a PNG and then laboriously figuring out the x/y locations of all the countries using the original SVG. Owie. On the plus side, I did somewhat extend the original SVG by wrapping the <path> elements for countries with multiple paths in <g> tags so you could use country code IDs for finding their respective <path> or <g> in the code for styling. A real shame it was too slow. Oh well.
Aside: A mysql trick
The MySQL section was also entertaining, figuring out partitioning and best indexing; culminating in the previous MySQL section - but I did find something else. When trying to do the IP address -> Country map I used a database of ip min/max ranges to country. The integer value of an ip address is the first number * 256 * 256 * 256, second number * 256 * 256, third number * 256 and last number as it is - sum them all together to get a unique number. When running a select on that table to get a between, though, it wasn't hitting indexes and was slow. The trick is to use LIMIT 1 on the statement and it hits the indexes. The explain plan proved it so... wow. Weird.
Anyway, still having a lot of fun coding this and figuring out little best practices for my app. I'm enjoing adding a status bar to the bottom of all the GUI apps I'm building that show progress, status, little notes and that sort of thing. Useful space, that footer.
