It's not about technology for its own sake. It's about being able to implement your ideas.
A Perl program I was working on last year had fifteen lines of code for loading and saving files wholesale (as opposed to going line by line). It could have been shorter, but I was using some system routines that were supposedly the fastest option for block reads and writes.
The advice I had been seeing in forums for years was that I shouldn't be doing any of this, but instead use the nifty File::Slurp module. It seemed silly to replace fifteen lines of reliable code with a module, but eventually I thought I'd do the right thing and switch.
I never should have looked, but File::Slurp turned out to be 800+ lines of code, including comments, and not counting the documentation block at the bottom. One of those comments stood out, as is typical when prefaced with "DEEP DARK MAGIC":
DEEP DARK MAGIC. this checks the UNTAINT IO flag of a glob/handle. only the DATA handle is untainted (since it is from trusted data in the source file). this allows us to test if this is the DATA handle and then to do a sysseek to make sure it gets slurped correctly. on some systems, the buffered i/o pointer is not left at the same place as the fd pointer. this sysseek makes them the same so slurping with sysread will work.
Still, I kept using it--the right thing and all--until one day I read about a Unicode security flaw with File::Slurp. Now I was using an 800 line module containing deep dark magic and security issues. Oh, and also no one was maintaining it. This was no longer the recommended solution, and there were people actively pointing out why it should be avoided.
I dug up my original fifteen lines, took out the optimizations, and now I'm back to having no module dependencies. Also the code is faster, likely because it doesn't have to load another module at runtime. As a footnote, the new right thing is the Path::Tiny module, which in addition to providing a host of operations on file paths, also includes ways to read and write entire files at once. For the moment, anyway.
(If you liked this, you might enjoy Tricky When You Least Expect It.)
permalink October 20, 2015
I'm James Hague, a recovering programmer who has been designing video games since the 1980s. Programming Without Being Obsessed With Programming and Organizational Skills Beat Algorithmic Wizardry are good starting points. For the older stuff, try the 2012 Retrospective.
Where are the comments?