LLVM just lost a contributor upset by the LLVM Community Code of Conduct. In my experience projects progress much faster when people are polite and decent. The fewer jerks on any team or platform, the better.
MTA for linux systems
I’ve mostly been using msmtp as a mail transport agent on my linux systems but I wanted to review what was available:
- nullmailer – 2.1 released 2017-Oct, nullmailer on github
- msmtp: 1.6.6 released 2016-Nov, msmtp on sourceforge
- dma – 0.11 released 2016-Feb, dma on github
- ssmtp – unmaintained
- esmtp – unmaintained
My preference is something hosted on github given the ease through which issues can be reported and tracked.
Meltdown and Spectre – And Not Buying Computers
I have never been in this position in my life. I have been an avid computer user since my family’s first 4.77 MHz 8088 PC circa 1986. This is the first time I’ve ever actively recommended people not buy a computer. The reason is that this is the first time all major microprocessors on the market have serious flaws that I believe should be resolved before purchasing. And that’s not the only issue.
Intel
It is interesting how almost all coverage refers to “Spectre and Meltdown” instead of “Meltdown and Spectre”. Meltdown is by far the more serious vulnerability and it affects all Intel microprocessors on the market and most Intel microprocessors produced since 2011 (possibly many from as early as 1995). Mitigations to this attack will likely reduce computer performance. This is not the end of the world but it is understandable that customers would be troubled both by the possibility of undetectable security failures and by the loss of performance to correct security. However, if this was all there was to the story I’d probably tell people to wait until the patches are in place and then resume buying. But I can’t.
The CEO of Intel, Brian Krzanich, after learning of these vulnerabilities immediately sold as much stock in the company as he could. That is insider trading. Given months to come up with a solution the company failed. Once the vulnerabilities were revealed Intel actively downplayed the significance of the vulnerabilities and engaged in a campaigned of misinformation to the public. This is a corporate culture that I cannot support. I hope Intel is able to find a path back to integrity, but I will not hold by breath. And I will not support a company with such an unethical business culture.
AMD
This should have been great news for AMD. Their microprocessors are not affected by Meltdown. However there have been reports of system instability for certain tasks on AMD microprocessors. This information has been difficult to pin down and it has been reported that the problem is fixed and that if a person encounters the problem that AMD will replace the chip with one that is unaffected. This is a good stance for a business to take. I suspect an AMD Ryzen system may be my next computer purchase.
The world is ready for a new generation of programming languages
I believe the rise of so many new programming languages in the past few years is in response to a reconsideration and reflection of what people like and dislike about current programming languages and not yet finding a language that is as great as they imagine a language could be. I suspect someone could write a book on this topic but I just wanted to document a few thoughts and notes.
- There is a renewed interest in performance (execution speed) and memory conservation. See: Rust, Go, Julia, Nim, Swift.
- Focus on safety and concurrency. See Rust, Swift, Go.
- More features from functional programming languages are being adopted to reduce side-effects such as immutable variables (at least optionally or by default as in Rust).
- “Composition over inheritance” – See Rust (no inheritance, all composition), Go (no inheritance, all composition), Julia
The languages being explored, not all are new:
- Rust
- Go
- Julia
- Nim
- Swift
- Elixer (purely functional)
- Clojure (purely functional)
The only thing I find missing is a focus on expressiveness. I would like to see some benchmarks that includes expressiveness as a metric. I really like Ruby because of it’s expressiveness. I am not a fan of purely functional languages but I like the idea of making functional concepts the default with optional ways to override when necessary, as Rust uses. I may try to come up with a benchmark to illustrate this issue. Mostly I think of this as a problem with large (think approximately 1GB in size) immutable arrays undergoing calculations. Plus, for loops with a variable for the current index or value is a pretty handy feature. I don’t prefer recursion as an alternative.
Links
- Some benchmarks of different languages
- The most promising languages of 2016
- The most promising languages of 2015 – Part 1 (Note: this is quite a similar list of changes in these new languages that I read after writing the above. I could have saved time just pointing to this.)
I created an account on Stack Overflow, then deleted it within a few hours
Like many developers I come across Stack Overflow pages with some frequency when performing research. I came across a question to which I had an answer and I decided to was time to try and contribute. Well, my answer was deleted, and then the question was deleted, and at no time did I get any information or feedback as to what was happening or why. This is poor user interface design. Though it may take some effort, it should be clear to users what is happening and why when they attempt to use a site. It only took one google query to find that my experience is hardly unique:
- The decline of Stack Overflow (2015 Jul) – This has a long list of similar articles at the end of the article.
- Why I refuse to use stackoverflow (2014 Sep-Dec) – This is missing from the above links and contains several stories.
- Why I no Longer Contribute to StackOverflow (2013 Dec) – This is in the above list but does a better job suggesting *why* Stack Overflow is problematic.
I wonder if most websites simply have a lifespan that plays itself out. This could be good as it would suggest that there is an evolutionary aspect to software. Stack Overflow successfully displaced the sites that came before it by doing what it does better, and something will come along and replace Stack Overflow and the sites occupying that niche now when a better design and implementation are presented. Maybe that next thing will work better for me. Or maybe this niche is just not something that I will interact with, and that’s fine too.
Disabling NetworkManager dnsmasq integration
I’ve disliked the NetworkManager dnsmasq integration for some time. As far as I understand it has caching turned off by default so I’m not sure what the rationale is at all. So disabling means 1 less system process and less magic on the system (since resolv.conf will reflect the actual nameserver settings rather than be overwritten by NetworkManager). The key to disabling this is to open:
/etc/NetworkManager/NetworkManager.conf
Comment or remove the line:
dns=dnsmasq
Then running:
sudo /etc/init.d/network-manager restart
And maybe:
sudo killall dnsmasq
Done.
Switching from dnsmasq to unbound
I’ve used dnsmasq for many years, more than a decade even, and I have no complaints. I think it works well as a lightweight DNS forwarding and caching server. But I have also been experimenting with unbound for a couple of years and recently decided to switch over my home network to use it. This is on an Ubuntu 12.04 server but it was pretty easy:
Remove dnsmasq:
apt-get remove --purge dnsmasq
Install unbound:
apt-get install unbound ldnsutils
Ensure DNSSEC is working (could use dig (dnsutils package) instead of drill):
drill com. SOA +dnssec | grep flags
drill sigfail.verteiltesysteme.net | grep SERVFAIL
drill sigok.verteiltesysteme.net | grep NOERROR
Here’s the config options I added to unbound.conf, other than settings specific to my network. I am testing with all harden options enabled to see if there’s any problems but YMMV:
harden-large-queries: yes
harden-short-bufsize: yes
harden-glue: yes
harden-dnssec-stripped: yes
harden-below-nxdomain: yes
harden-referral-path: yes
hide-identity: yes
hide-version: yes
prefetch: yes
prefetch-key: yes
root-hints: /etc/unbound/root.hints
To get the root hints I created a script in cron.monthly that looks like this:
#!/bin/sh
/usr/bin/curl -sS -o /etc/unbound/root.hints https://www.internic.net/domain/named.cache
So no root hints will be updated every month. The only other thing I may consider at some point is running in chroot but for now this was pretty quick and easy. Also to look at the stats:
watch unbound-control stats_noreset
Reducing memory usage of rsyslog
I’ve never been a big fan of rsyslog, preferring syslog-ng instead. However, I am giving rsyslog another chance. One of my biggest problems with rsyslog has been it’s high memory usage, documented by many people in many complaints that can be found using google. Now this emmory is usually, though not always, virtual and not resident. That means rsyslog is not actually hogging memory but merely causes reporting problems when searching for processes with high memory usage as it will often show up. However, I’m trying a new solution that I hope will improve the situation. From the rsyslog wiki:
rsylsog is a (potentially massively) multi-threaded syslogd. Each of the threads requires a runtime stack. Rsyslog uses no specific stack allocation and sticks with the OS default. Seen in practice have stack allocations of 8 to 10 MB per thread. In a process trace, this can look like a memory leak.
So how big is the default stack for rsyslog:
cat /proc/`pidof rsyslogd`/limits
On all of the 64-bit Ubuntu systems I’ve tested the answer is:
Max stack size 8388608 unlimited bytes
So 8MB. The easiest fix I discovered is this:
Edit /etc/default/rsyslog and add the line:
ulimit -S -s 128
This sets the soft limit for stack size to 128K. Then restart:
restart rsyslog
Then verify:
cat /proc/`pidof rsyslogd`/limits
Max stack size 131072 unlimited bytes
Stumbling on cool things
I’m interested in emulators, always have been, and I was wondering if anyone had used LLVM to write an emulator. I came across this highly detailed account which was a fun read:
Statically Recompiling NES Games into Native Executables with LLVM and Go
It concludes stating that static recompilation is probably less practical than JIT, which I think is the correct conclusion. Still, nice to have someone go through the effort of building something just to show how it can be done.
Amazon’s new TLS library
I’m liking what I see, good design principles, aiming for simplicity and reviewability. I would not be surprised to see this become a huge success as people eschew the extra functionality of OpenSSL (and others) in favor of software with less security risk. Amazon’s backing is likely to propel s2n over other implementations with similar goals.