I had made a home in Algeria; I was happy with my life and my work in the national press. In 1969, events took an extraordinary turn. Late one night I received a call from Charles Chikerema, the representative of the Zimbabwe African People's Union, one of many African liberation movements with an office in the city. He told me that the Black Panther Eldridge Cleaver was in town and needed help.–Short memoir by Elaine Mokhtefi about working with the Black Panthers in Algeria. Interview with Mokhtefi (audio and transcript).
We've all encountered database tables that look like this:
ID Data ----- -------------------------------------------- 00001 TRUE, FALSE, FILE_NOT_FOUND 00002 MALE|FEMALE|TRANS|EUNUCH|OTHER|M|Q|female|Female|male|Male|$ 00003 <?xml version="1.0" encoding="UTF-8"?><item id="1234"><name "Widget"/>...</item> 00004 1234|Fred,Lena,Dana||||||||||||1.3DEp42|
Oh the joy of figuring out what each field of each row represents. The fun of deciphering the code that writes and reads/parses each row of data. In a moment, you will fondly look back on that experience as the Good-Old-Days.
The task of administering elections in the Great White North is handled by the appropriately-named agency Elections Canada. As part of their mandate, they provide the results of past elections in granular detail, both as nicely formatted web pages and as downloadable raw files. The latter are meant to be used by researchers for studying how turnout varies across provinces, ages, races, etc., as well as arguing about the merits of proportional representation versus single transferable votes; and so forth.
One of the more comprehensive data files is descriptively known as Table-Twelve, and it contains a record for every candidate who ran in the election. Each record contains how many votes they got, the riding (electoral district) in which they competed, their affiliated party, home town, occupation, and hundreds of other details about the candidate. This file has been published for every election since the 38th general in 2004. Vicki was charged with creating a new parser for this data.
Table-Twelve is a CSV file in the same way that managers describe their new agile process as <details of waterfall here>. While parsing a CSV file in general is no big deal, writing a function to parse this data was far harder than she expected. For one thing, the column titles change from year to year. One might think Who cares, as long as the data is in the same sequence. One would be wrong. As an example, depending upon the year, the identifier for the electoral district might be in a column named "Electoral District Name", "Electoral District" or "District", and might contain a string representing the district name, or a numeric district identifier, either of which may or may not be enclosed in single or double quotes. Just to make it interesting, some of the quoted strings have commas, and some of the numbers are commafied as well.
Further inspection revealed that the columns are not only inconsistently named, but named so as to be completely misleading. There's a column labeled "Majority". If you're thinking that it contains a boolean to indicate whether the candidate got a majority, or 50%+1 of the number of cast votes (i.e.: "How many votes do you need for a majority?"), you'd be mistaken. Nor is it even a slight misuse (where it should have been "Plurality"). Instead, it's the delta between the winning candidate and the second-place candidate in that riding. They also helpfully give you the quotient of this delta to the total cast votes as the "Majority Percentage".
Canada has a parliamentary system; it's also important to know how many candidates of each party won, so the party designation is obviously going to be easy to access, right? Or maybe you'd like to sort by surname? Well, it turns out that the party is appended to the field containing the candidate's name, delimited with a single space (and possibly an asterisk if they were incumbent). But the candidate's name and the party are already each a variable number of words (some have middle names or two surnames) delimited by single spaces. The party name, however, must be given in both English and French, separated by a forward slash. Of course, some parties already have a slash in their name! Oh, and if the candidate didn't run as a member of a party, they might be listed as "Independent" or as "No affiliation"; both are used in any given file.
Above and beyond the call of making something difficult to parse, the files are full of French accented text, so the encoding changes from file to file, here ISO-8859, there UTF-8, over there a BOM or two.
Don't get me wrong, I've written parsers for this sort of garbage by creating a bunch of routines to do trivial parsing and using them for larger logical parsers, and so on until you can parse all of the fields in an entire row, and all the special cases that spew forth. But the files they were supposed to parse were consistent from one day to the next.
Vicki is considering pulling out all of her hair, braiding it together and using it to hang the person who designed Table-Twelve.
I’ve added two new case studies to Ansible for Networking Engineers online course:
Create network diagrams from LLDP information playbook focuses on creating a single summary report based on information from numerous devices (and the report just happens to be network diagram in DOT format).Read more ...
Mastering Layers: Beginner Techniques with Dave Cross
Join Dave Cross to learn how to become successful using Photoshop’s layers. Aimed at beginners, this class is designed to help you understand what a layer is, discover what types of layers exist and the ways they can be used, navigate the Layers panel, learn how to manage your layers, and much more. Understanding layers is the key to unlocking Photoshop’s tremendous capabilities for all manner of creative pursuits. By the end of this class you’ll have the confidence to take on more challenging projects.
In Case You Missed It
Learn how the top commercial advertising pros create stunning composite portraits, with Joel Grimes and Matt Kloskowski. In this part one of a two-part series, Joel takes you behind the scenes and walks you through every aspect of his creative process. By the end of the class you’ll know exactly how the pros do it, and learn to shoot your portraits with more creativity and confidence than ever before!
Because the last time it was done, it was not forbidden, because good jobs cluster in only a few regions now and because of vast influxes of foreign money, we have charts like this:
So, almost a 100% increase in five and a half years. (People living in Vancouver wish housing prices had only risen this much.)
Meanwhile, the Fed is muttering to itself about how there is almost no inflation, because they don’t measure housing price increases as inflation and consider the most important inflation that which does not include energy and food.
In other words, if the price of having a home, staying warm in your home or cool, driving your car and feeding yourself is going up, well, that’s just not very important.
A lot of people got very rich in real estate speculation and mortgages and downstream securities last time, and the vast majority of the rich ones got to keep the money they made. Even those who lost it, were mostly made whole by government. (Ordinary home owners were, uhhh, not made whole.)
Given it worked last time, given there was no real penalty for doing it and that the Fed and other central banks proved they were willing to bail out the rich to the tune of trillions of dollars, why not run the play again? The profits are privatized, the losses at the end will be socialized. Heck, with a bit of luck the Fed will print money pre-emptively to make sure that there is never a crisis for rich people ever again, just every increasing asset prices.
(This applies to the stock market as well.)
There is, mind you, a real economy buried under all the money being funneled to rich people somewhere, and at some point that economy may just collapse. After all, all the people who own these fancy condos and houses expect a servant class to take care of them.
But perhaps that can all be turned over to robots, as Silicon Valley wants, and the poor can just be expelled from places like SoCal, DC, New York, Vancouver and Toronto entirely, to slowly drug themselves to death, or perhaps just starve, in the vast interior wastelands of the continent where “real” people don’t want to live.
This is, fairly explicitly, what Silicon Valley tech bros want, to not need surplus people.
I wonder, though, how many of them will find that they too, are surplus, when AI becomes able to code and write ads.
It will, at least, be amusing.
The Ware for June 2017 is an ultrasonic delay line. Picked this beauty up while wandering the junk shops of Akihabara. There’s something elegant about the Old Ways that’s simply irresistible to me…back when the answer to all hard problems was not simply “transform it into the software domain and then compute the snot out of it”.
Grats to plum33 for nailing it! email me for your prize.
Most of my replacement parts arrived, including the 1626 tube, which I soldered into place. I do have two problems that will require more part orders. First, since I am reverting the design back to the 1629 magic eye tube from the 6E5, it switches the required tube socket from the six pin part I had to remove and an octal socket for the 1629.
Second, part of the first order was wrong. I remember that when I found the 22K 2W resistors on Digikey, I initially ordered 6, the quantity I needed. However, the web site popped up an alert telling me that for the same price I would get 10, because of the volume break at that number. I approved the change but didn't check closely to see that it had changed the type of resistor to a 1/4W 10K that I don't need.
When I was ordering the 22K resistors today, the same pop up was delivered but I cancelled out of it and manually ordered 10 of the desired part. It was now correct and will arrive later this week or over the weekend.
IBM 1401 RESTORATION WORK
We have been fighting erratic reading on the 1402 reader that is part of our "Connecticut" system over the past couple of weeks, but today someone noticed that dust was wedged into slots in a plastic disc used with a solar cell to generate the timing pulses for the 12 card rows. With an air compressor and vacuum, the disc was cleaned and our reader is back to proper operation.
We still have a permanent error in core at one address (high in memory, fortunately). We will try varying the core unit voltage in both directions to see if we are right at the edge and failing on the weakest core, or if we have a more pernicious flaw in that core location.
If we can't get the core working, we have a spare 4K core unit that we can swap in. It is a major task, but much easier than disassembling and rethreading a core plane to replace a bad core.
Somewhere around January of 2014, I discovered Kerbal Space Program.
This was my first successful craft, which lifted off, flew a short distance from the pad, then (after several botched attempts and the deaths of several Kerbal pilots) touched down.
This was followed by a hideously inefficient, yet perversely pleasing design, combining three off-axis tanks and rocket motors in a tetrahedral configuration.
For obvious reasons, this design didn’t make it far, and I decided to pursue more pedestrian goals, such as walking on the moon, or, as Kerbals would have it, “Mun”.
Early solid-booster designs failed spectacularly, but eventually we got a lander to Mun.
An unplanned interaction between the landing gear and the rocket motor base of the descent stage destroyed one of the seven nozzles for the lander. We were forced to shut down the corresponding engine and funnel fuel to the remaining four balanced motors.
As usual, the deceleration and landing burns used up far more fuel than anticipated, and we were forced to land only the command module. Luckily, the command module had landing gear–intended for earth–which worked just fine on the Mun.
Unfortunately, this left us with the small problem of getting Jeb Kerman back home to Kerbin. The command module was able to reach 21 kilometers altitude, but didn’t have enough fuel to complete an orbital injection burn. Thinking quickly, Jeb jumped out of the control module and used his EVA jets to complete the insertion burn. The command module fell back to Mun and exploded, leaving Jeb stranded in Mun orbit.
With only limited parts, a rescue mission required creative construction. This design forced the rescued Kerbal to hang suspended, upside down, in his command chair, while thinking about what he (or rather, the incompetent nincompoops over at Fab & Mission Control) had done.
With Jeb’s EVA tank exhausted, we were forced to (gently) smash the entire rescue ship into Jeb, hoping that he could cling to the handholds on the recovery capsule.
Re-entry and landing went smoothly, and Jeb returned, rather shaken, to Mission Control.
Following the Great Fuel Debacle, our engineers discovered spiral staging, and proceeded to design, and explode, significantly higher delta-V launch vehicles.
Subsequent landings on Mun and Minmus went (comparatively) smoothly.
As soon as we’d performed enough Scientific Research to build more sophisticated parts, I decided to launch a space station. This is the first space station module, being delivered to geosynchronous orbit approximately 200 meters south of the launch pad.
Launching asymmetric objects is hard. You have to balance atmospheric drag (in atmosphere) and center-of-mass (in both atmo and space). Eventually we got it right. Here’s the first hab node and research lab, in a more suitable orbit.
Space stations need spaceplanes to deliver and recover crew. Of course, I’m not skilled enough to build actual spaceplanes. This one uses regular rockets to get into orbit. Once there, it can shuttle crew between orbiting ships and stations, and return them to earth when needed. With several Kerbals left floating, empty-tanked, in orbit, this seemed like an important capability for our burgeoning space program.
With our capabilities constrained by a lack of science, I set out to seed the solar system with instrumented probes. Here’s a seven-probe seed pod, which scatters its probes across a planet’s surface.
With new research came new parts, and even larger launch vehicles. This one provides about 10,000 m/s of total delta-v; more than enough to put a crewed lander onto Mars' (Duna’s) moons. I don’t think I had fuel lines at this point, so it’s not even spiral staged. The entire first stage goes straight to orbit with ~6000 m/s to spare.
The interplanetary stage carries ~3000 m/s to get us to Duna and perform the appropriate insertion burns.
To leave space for the docking port, we’re limited to these tiny side-mounted motors. So long as we keep to low-gravity environments (e.g. very small moons) they’re very efficient, leaving us ~1800 m/s for deorbit, landing, ascent, and docking with the interplanetary transfer vehicle.
To save landing mass, only the transfer vehicle carries Reaction Control motors and monopropellant. We pilot it via radio and dock it to the lander.
… which allows the entire crewed science lander, and its valuable experiments, to return to Kerbin intact.
That is not to say that all missions went according to plan.
Tomorrow: we have to go bigger.
What I'm going to call system continuity for my lack of a better phrase is the idea that some of the time in some places, you can trace bits of your current environment back through your organization's history. You're likely not still using the same physical hardware, you're probably not using the same OS version and possibly even the OS, the software may have changed as may have how you administer it, but you can point at elements of how things work today and say 'they're this way because N years ago ...'. To put it one way, system continuity means that things have a lineage.
As an example, you have some system continuity in your email system if you're still supporting and using people's email addresses from your very first mail system you set up N years ago, even though you moved from a basic Unix mailer to Exchange and now to a cloud-hosted setup. You don't have system continuity here if at some point people said 'we're changing everyone's email address and the old ones will stop working a year from now'.
You can have system continuity in all sorts of things, and you can lack it in all sorts of things. One hallmark of system continuity is automatic or even transparent migrations as far as users are concerned; one marker for a lack of it is manual migrations. If you say 'we've built a new CIFS server environment, here are your new credentials on it, copy data from our old one yourself if you want to keep it', you probably don't have much system continuity there. System continuity can be partial (or perhaps 'fragmented'); you might have continuity in login credentials but not in the actual files, which you have to copy to the new storage environment yourself.
(It's tempting to say that some system continuities are stacked on top of each other, but this is not necessarily the case. You can have a complete change of email system, including new login credentials, but still migrate everyone's stored mail from the old system to the new one so that people just reconfigure their IMAP client and go on.)
Not everyone has system continuity in anything (or at least anything much). Some places just aren't old enough to have turned over systems very often; they're still on their first real system for many things and may or may not get system continuity later. Some places don't try to keep anything much from old systems for various reasons, including that they're undergoing a ferocious churn in what they need from their systems as they grow or change directions (or both at once). Some places explicitly decide to discard some old systems because they feel they're better off re-doing things from scratch (sometimes this was because the old systems were terrible quick hacks). And of course some organizations die, either failing outright or being absorbed by other organizations that have their own existing systems that you get to move to. Especially in today's world, it probably takes an unusually stable and long-lived organization to build up much system continuity. Unsurprisingly, universities can be such a place.
(Within a large organization like a big company or a university, system continuity is probably generally associated with the continuity of a (sub) group. If your group has its own fileservers or mail system or whatever, and your group gets dissolved or absorbed by someone else, you're likely going to lose continuity in those systems because your new group probably already has its own versions of those. Of course, even if groups stay intact there can be politics over where services should be provided and who provides them that result in group systems being discarded.)
In celebration of the launch of the “Great 78 Project” the Internet Archive is sponsoring a Community Screening of the PBS documentary series “American Epic”, an inside look at one of the greatest-ever untold stories: how the ordinary people of America were given the opportunity to make 78 records for the first time.
“Without the recording lathe, Willie Nelson would have never heard the Carter Family sing. Neither would Merle Haggard or Johnny Cash. These portable machines toured the country in the 1920s, visiting rural communities like Poor Valley, West Virginia, and introducing musicians like the Carter Family to new audiences. This remarkable technology forever changed how people discover and share music, yet it was almost lost to history until music legend T Bone Burnett and a few friends decided to bring it back.” Charlie Locke – WIRED
The program will be introduced by Brewster Kahle of the Internet Archive.
Please RSVP on our free Eventbrite page.
Date: Sunday July 30th – “The Big Bang” (:54 min) & “Blood and Soil” (:54 min)
Date: Sunday August 6th – “Out of the Many, the One” (1:24min)
Time: Doors Open at 6:30 pm – Screening(s) at 7:00 pm
Cost: FREE and open to the public
Where: Internet Archive Headquarters 300 Funston Avenue, San Francisco, CA
“American Epic” Teaser: https://youtu.be/jcbATyomETw