Week of 2008-11-08 17:00 to 2008-11-15 16:59

Web log analysis and statistics

There are a number of open source web stats programs out there. Many have been around for quite some time. The big 3 are Analog, Webalizer, and AWStats. I used to be a fan of the Analog + Report Magic combination. However Analog is now quite out of date. Webalyzer was recently updated after 4 years of stagnation. I tried out AWStats which has stayed current but found its setup to be cumbersome. In looking for alternatives I came across a number of newer, and often smaller, projects. The one I tried tonight was Visitors and I am extremely satisfied with it thus far. It was trivial to setup and the results were helpful. It focuses on per IP (Visitor) stats while ignoring some other stats which would be nice like overall page hits and traffic graphs. I’m hoping that it plus Piwik might be a good solution for most websites, though I recognize that AWStats, though unfun to configure, might still be necessary to meet some needs.

Rails Sessions Work

While working on a Rails app I came across the fact that the sessions database table was accumulating 20,000 or so entries per day. Since most of these were from search bots I figured that it would be best to find a way to avoid having so many entries. I quickly came across the fast_sessions plugin, which I had noted in the past. I tried implementing it as a replacement for SQL Session Store but it failed with MySQL connection errors. So that having failed I decided to examine what was happening with sessions in more detail.

NAT Reflection on pfSense firewall

A client had a firewall where the machines on the LAN couldn't access services being port forwarded from the public IP to a machine on the LAN when using that public IP. In order to enable this (which in shorewall is the routeback option in the interfaces file if I remember correctly) I went to "System - Advanced" and unchecked/disabled "Disable NAT Reflection" thereby enabling NAT reflection. I'm not sure if it's an ideal solution but it was a nice quick change to get the functionality they wanted.

Wanting a better fdupes program

The fdupes program came in handy today when having to delete (or move, to be safe) a large number of duplicate files. I used this command:

  • fdupes -r -n -1 . | cut -f1 -d' ' | xargs -I '{}' mv -v '{}' /var/tmp/

However I still had to run this command several times in various subdirectories because I didn’t want to delete matches that were in different directories. So 1 nice change would be to not find matches in different directories when using recursion. Another nice option would be to automatically delete based on a small set of criteria (probably older or newer). I can imagine the improved version would look something like this:

  • fdupes -r -n --delete-newer --donotmatchacrossdirectories .
Creative Commons License Except where otherwise noted, content on this site is licensed under a Creative Commons by-nc-sa 3.0 License