Planet Linux Australia

Syndicate content
Planet Linux Australia -
Updated: 19 min 56 sec ago

Hamish Taylor: In an ideal world … how to change my address

Fri, 2015-10-23 14:28

Recently I moved house.

I  hate moving. Not just for the having to pack everything into boxes at one end then then unpack everything at the destination (which for this move I didn’t have to do!), but mostly because I have to go through the pain that is changing my address.

It turns out that I interact with a lot of organisations, from finance institutions (banks, credit card companies, car insurance, house insurance, health insurance, etc), to official organisations (driver licencing, Medicare, electoral, organ donor register, etc), to community (Red Cross blood donor, 3RRRFM, etc) and mundane organisations (Costco, etc). And that’s just a fraction of them.

I was thinking that, rather than having to fill in what feels like a million forms and waste time that could be spent being a productive public servant or dad for my kid, why isn’t there a central contact details database that I update once? I’m sure that smarter minds than mine have considered this, but I think an opportunity exists for some organisation (government or private) to do this. In the day and age of ‘over-sharing’, are people still averse to putting their address, phone number and email details into a central database? Login security could be addressed using two-factor authentication, such as used by Google Authenticator, or sending a one-time code via SMS or email.

Many services, such as Twitter and Facebook, are set up to authorise other apps to access them. An example of this is when I used my Facebook account to sign up for Freecycle which operates as a Yahoo Group.  I ‘authorised’ Facebook to talk to Yahoo. I’ve also authorised Twicca on my Android smartphone to talk to my Twitter account.

In the same way, in this theoretical single contact details database, I could let the various companies and organisations that I interact with, access my updated contact details. Maybe they could poll this database once a week to look for updated details. I understand they’d have many different backend CRM systems so there may be some manipulation required, but nothing that’s too hard to fix with a bit of scripting.

I could also remove their access when I cease using their services. If I’m not longer banking with Bank A, then I revoke their access so they can’t find out how to contact me.

Does this sound sensible or silly? If sensible why hasn’t Google or someone done this already?

Hamish Taylor: Idea from BarCamp Canberra #barcampcbr

Fri, 2015-10-23 14:28

Yesterday I went to the second half of BarCamp Canberra 2012 (I was busy in the morning and couldn’t make it).

As per usual for a BarCamp there were many great ideas being discussed. Someone (Craig?) suggested that we all go home and write blog posts about our own great ideas.  So here goes …

My ideas is this: to build a website to facilitate the transfer of mobile phone credit from people who have a surplus to people who need it.

My wife and I are currently using Telstra pre-paid and every so often when it gets near the expiry date, if there’s any unused credit we transfer some (or all) of that to the other account. Telstra call this ‘CreditMe2U’ and my understanding is that it can be used on any post- or pre-paid accounts. There’s a few limitations, such a maximum of $10 per day and some limit per month.

I see the site facilitating someone posting up that they need, say $5 credit. Anyone should be able to do this for any reason. The request could be as little as just a phone number and an amount.

Someone else, who has surplus credit, would transfer them some credit from their account, and then mark that the transaction has happened. This ensures that the requester doesn’t get flooded with credit transfers and multiple people who have surplus credit don’t end up  helping just one person. The requester would also not be able to make another request for 24 hours (based on phone number).

I would be reluctant to require people to register for accounts, as I think that would kill it entirely. It should be able to be truly anonymous. I would also be really keen to see that the site is not indexed in any way (robots.txt, exclusions, etc), so that numbers can’t be linked with requests.

I’m not sure if carriers other than Telstra have this option, but it’s worth investigating.

While there would be obvious ways to ‘game’ this system, and it’s not a fully thought through idea, it could become so with some feedback. So, what do you all think?

Hamish Taylor: Fun with JavaScript!

Fri, 2015-10-23 14:28

Hoping someone can help me with this JavaScript problem. I’m trying to pass an array to a getElementById with the purpose of making multiple cells in the table take the class. I can get it working with one array location but not with more than one. Please help!

<!DOCTYPE html>





function changecolors(redsarray,yellowsarray,greensarray,graysarray)


var redsarray = new Array();



var yellowsarray = new Array();



var greensarray = new Array();



var graysarray = new Array();









<style type=”text/css”>

.red {background-color:red;}

.yellow {background-color:yellow;}

.green {background-color:green;}

.gray {background-color:gray;]




Content …

<table border=”1″>


<td id=”r1_c1″>



<td id=”r1_c2″>





<td id=”r2_c1″>



<td id=”r2_c2″>





<td id=”r3_c1″>



<td id=”r3_c2″>





<td id=”r4_c1″>



<td id=”r4_c2″>




<button type=”button” onclick=”changecolors()”;>Button</button>




Hamish Taylor: Follow up: The woeful state of communications in Australia’s capital city

Fri, 2015-10-23 14:28

In January 2011, I posted about my experiences in trying to get an internet connection provisioned at my new home

I am now posting from our Internode naked DSL connection. To be honest, this has been working for many months, I have been slack in posting this follow up!

The Telstra guy did come back and install the line. But only after we ordered a full phone line, dial tone and all, at around $30/month. Not to mention the $299 installation fee.

After that was installed, Internode activated the ADSL. Even that took multiple calls to get the technicians back to the exchange as things went wrong.

After that was all sorted out, it was then converted to a ‘naked ADSL’ service. Effectively cancelling the dial tone service.

The rampant stupidity of the Australian communications system is truly breathtaking. And expensive. What should have been a very simple thing to get going – a naked ADSL line – proved to be extremely difficult and expensive.

But now we have Internode naked ADSL and NodePhone. Finally.

(As an interesting side note, we retained our Melbourne based phone NodePhone (VoIP) number. When the Mitchell chemical fire occurred the other day and half of Canberra was on alert, we received a call on the VoIP number, as it is registered at this address. Both mine and my wife’s mobile phones are through Optus, also registered at this address and didn’t get an SMS or call. Either the emergency alerting system or Optus messed up there. I’d be guessing the latter.)

Unfortunately, we are so far away from the exchange that we only get around 500 KB a second (half a MB a second). Back in Melbourne, close to the exchange, I was getting 2.2 MB a second, so around four times faster).

But at least we have it

OpenSTEM: Mirobot v2 Robotics Kits and Soldering Kits Available

Thu, 2015-10-22 18:30

The Mirobot v2 logo turtle robotics kits will be here shortly. These are the updated version of the kits we have been using at primary schools (year 4-6) this year in our Robotics and Programming workshops. The new model doesn’t require little pegs any more, the structure now holds itself together with a beautiful designed slot mechanism. Kudos to Ben Pirt for an awesome design!

The robot frames are made of lasercut MDF, and the circuit board is Arduino controlled. All aspects of the design is open and available. The robot can be used to draw, but now also comes with bump sensors and line following capabilities. Communication is through wifi over a raw or web socket. There are a number of programming and control options, from Scratch-style visual systems to a brand new Python library!

By default the v2 comes with a pre-soldered circuit board, but especially for OpenSTEM Ben is offering a non-soldered PCB so we can continue doing the soldering part with classes also. We have found this to be both a great enabler for students, as well as teach that people can build things almost from scratch. But you choose… we keep both the soldered and un-soldered kits. Either way, this is a great project to do with your kids at home, quite a few parents of students that do our workshops also continue in this way.

If you order now, we’ll still be able to include you in the first shipment!

Now for Electronics Soldering! If you or your children want to also do some soldering but don’t have the necessary tools yet, we now have sets available. We assemble our own classroom soldering kits ourselves from a number of sources, as sets found in shops have flimsy or awkward stands. We use a solid steel stand, that also features a wire cleaning ball – this works much better than a wet sponge and it is much easier to maintain. We also include a number of other useful items.

You can order the soldering kit together with a Mirobot kit, or on its own.

Shipping of orders including Mirobots will be in November. This is likely to be our final Mirobot order this side of Christmas, so we do recommend you order now if you want to have the kit available over the holidays.

James Morris: LSM Mailing List Being Archived Again

Thu, 2015-10-22 16:27

Several folks noticed that all of the known LSM mailing list archives stopped archiving earlier this year.  We don’t know why and generally have not had any luck contacting the owners of several archives, including marc and gmane.  This is a concern, because the list is generally where Linux kernel security takes place and it’s important to have a public record of it.

The good news is that Paul Moore was finally able to re-register the list with, and there is once again an active archive here:

Please update any links you may have!

David Rowe: Modems for VHF Digital Voice

Thu, 2015-10-22 12:30

I’ve been thinking about modems for a VHF FreeDV mode. The right waveform and a good demodulator is the key to high performance. However it would be nice to make some re-use of existing FM VHF radios. So is it possible to come up with a waveform that can pass through legacy FM radios, but also be optimally demodulated with a SDR?

My first guess was that the problem with legacy radios is the 300Hz High Pass (HP) pass filtering. So I came up with a waveform with has no DC. Brady pointed out this was Manchester Encoding (ME), used in all sorts of applications for just this problem. Each data bit is Manchester encoded to two bits, so a 2400 bit/s bit-stream becomes a 4800 bit/s bit-stream that is then 2FSK modulated. Turns out the ME-2FSK signal doesn’t have much low frequency energy so passes happily through the audio pass band filtering of regular FM radios.

Here is a block diagram of the idea. We have the option to demodulate the signal using a legacy analog radio or, with higher performance, an optimal FSK demod:

This is what the spectrum of the ME-2FSK looks like at the output of the analog FM demodulator before high pass filtering. Notice how there is not much energy beneath 300Hz? So we are not going to lose much due to the 300Hz HP filter.

Here are the time domain modem signals before and after the 300Hz High Pass filter. Pretty similar.

The ME-2FSK scheme works OK in my simulation, so I think it’s possible to squirt 2400 bit/s through a $40 HT with acceptable modem performance using 2FSK. This means we can do VHF FreeDV using your laptop/SM1000 and a $40 radio, and it will work just as well as existing VHF DV modes, and even pass through analog repeaters.

Real gold would be a way to send 4FSK through a HT, that (if you have a SDR) can be optimally decoded at a much lower Eb/No. Unfortunately I couldn’t work out how to do that. For optimal 4FSK you need the tones spaced at the symbol rate Rs. This means -1.5Rs, -0.5Rs, 0.5Rs, 1.5Rs, which won’t fit into 5kHz deviation with Rs=4800. So how about Rs=2400? Well when I tried Rs=2400 through the FM demod the modem appears to be 3dB worse that Rs=4800. I’m not sure why. Possibly deviation, as I get the same results with the 300Hz HP filter removed. Or maybe I messed up the simulation. Oh Well. Working backwards, this suggests one reason the ME 2FSK waveform works so well at Rs=4800 is greater deviation.

Moving to the optimal 4FSK demod approach, here are the outputs of each filter from an optimal 4FSK demod. The pretty colours represent the different filter ouputs. The lower plot is the decimated filter outputs, after sampling at the ideal timing instant.

I’m inclined to use both 4FSK and ME-2FSK. We could run ME-2FSK on links with legacy radios and 4FSK on SDRs that support optimal demodulation. That 6dB Eb/No for optimal 4FSK, combined with Codec 2 running at a lower rate, is a huge gain over current analog and DV systems.

Summary of Candidate VHF Waveforms

I’ve now played with quite a few modem waveforms, and have compared them in the table below. Eb/No is for a BER of 2%, which is roughly where Digital Voice codecs fall over. There are two Eb/No figures, one for an ideal demodulator, the other when using a demod that works through a legacy FM analog radio.

Waveform Eb/No (ideal) Eb/No (FM) Comment Read More PSK 3.0 na requires linear PA, complex coherent demod GMSK 5.0 9.0 requires “data” port, complex coherent demod [1] [2] 4FSK 6.0 na simple demod, good fading ME-2FSK 8.5 12.0 simple demod, good fading, $40 HT! DMR 4FSK na 11.0 standardised [3] AFSK-FM na 16.0 As used in APRS [4]

The complexity of the demods required for coherent PSK and GMSK is not a show stopper, as we only have to write GPL modem code once. However coherent demodulation means other sources of “implementation loss” such as phase recovery that make the ideal performance hard to achieve. Non-coherent mFSK is rather simple in comparison, we just need a fine timing estimator. Less to go wrong. No phase estimation means fading will have less impact than coherent PSK/GMSK. Fine frequency offsets won’t bother us. mFSK is, however less bandwidth efficient.

GMSK coherently demodulated or through a legacy FM radio looks pretty good, but does require a “data port” with unfiltered access to the FM modem. So no $40 HTs.

Note the distinction between ideal non-coherent 4FSK, and the 4FSK modem used by DMR and similar Digital Voice modes like C4FSK. The latter are not optimal waveforms, and in our simulations under-perform by around 6dB. We can’t find any explanation of why these waveforms were chosen for DMR or C4FM. I am guessing that have been developed with the specific use of legacy FM radio architectures or reduced RF bandwidth in mind.

Running the simulation

I set up a bunch of simulations of various combinations so they all have about 2% BER:

octave:224> mancyfsk

Rs=4800 2FSK ideal demod

  EbNodB: 8.5 BER 0.023

Rs=4800 2FSK analog FM demod, not too shabby and pushes 2400bit/s thru a $40 HT!

  EbNodB: 12.0 BER 0.021

Rs=2400 2FSK analog FM demod, needs more power for same BER!  Che?

  EbNodB: 15.0 BER 0.027

Hmm, doesnt improve with no 300Hz HPF, maybe due to less deviation?

  EbNodB: 15.0 BER 0.027

Rs=2400 4FSK ideal demod, nice low Eb/No!

  EbNodB: 6.0 BER 0.025

Further Work

It would be great to test the work above in the real world, for example get the ME-2FSK modem software into a form that we can do calibrated noise (or MDS) tests on a real FM radio.

Binh Nguyen: More Eurasian/Middle-East Geo-Politics - Part 2

Thu, 2015-10-22 05:12
- if it wasn't for the fact that so serious (and both sides were so duplicitous with regards to spying) it would be funny. I doubt they will be able to figure out a hands off areas. One thing an area of common ground would be how far they can actually push things though. For instance, you can't shut down essential utilities for more than a particular period of time, you shouldn't shut down hospital systems, etc... during a period of non-conflict

- imagine if you homeland were Russia or China would you discourage your coutry men from trying to earn a living? What would you do?

- a lot of intellectual property is being generated by China and Russia. They are not pure stealers of intellectual property as some people would like you to believe. In a lot of cases China is way up there as are other Asian/European countries who are considered to be not particularly creative or industrious

- some background on corruption in Russia if anyone were interested. I think the core problems are that their culture and system essentially makes the assumption that those in charge will be 'clean' and indoctrinated. Without a good system of 'checks' and 'balances' the system collapses in on itself. Just like a lot of other behaviour the most extreme profits the most so the mean moves towards the outliers

'Corruption is endemic in Russia'

Popular Corruption & Russia videos

- the problem is that both with Russia (and China) is that even if they are able to shed their 'reputation' people/countries are still going to look at them suspeciciously. Listen into interviews from the West and even journalists seem to be frustrated that Russia and China don't seem to convey what the feel to be their true intentions. One thing I remember is how someone from Russia once said that Soviet propaganda was silly and dishonest that it came across as 'funny'. That's not the case with modern Russian propaganda which is a confusing mish-mash. Of late, they've been telling lies until they've completed their objectives at which they tell the truth...

- more background regarding Putin for those that are interested

Professor Stephen Cohen on Obama and Putin UN Meeting

Putin Issues Ultimatum to Turkey - Stop Supporting ISIS or Face Break in Diplomatic Relations

Putin - Obama and Hollande are not Syrian citizens, can't decide Syria's future

Putin's Growing Tensions With The West

VEVO Documentaries _ Putin's Way _ Documentary Movie

CrossTalk - Putin's 15 Years

U.S persists in misunderstanding Putin

- some interesting Middle East background videos (think carefully. Not everything on the Interwebs is real, I sometimes listen to these and other news while working on other stuff). Obviously, it doesn't really matter who is in charge of world affairs, geo-politics will always play a significant role

Instability in the Middle East - Institute of Politics

Middle East Documentary 2015 _ Mind Blow Manipulative & Betrayals History 720 HD

- what I don't understand about many economies is that in spite of being supposedly religious many industries use simultanouesly harmful and addictive substances/systems. Think cigarettes (nicotine), drugs (can even cover medicines), fast food (sugar), beverages (mostly sugar and caffeine), consumption (facilitated through round the clock marketing), etc... All of these over time can actually kill you or severely degrade the quality of your life or cause severe problems with the local economy. In fact, in Russia they had a Vodka problem so severe that it was causing population growth issues (through cardiovascular difficulties amongst men)... I wonder how much these systems have had upon population growth throughout the world? I'm curious how countries control this if they are causing major problems, the obvious ones are high taxation or banning... I'm interested in other novel approaches

- the great irony of the US economy is that in giving in to 'freedom' for the individual private companies have now grown so large that they rival the government's power. Recently it was said that due to the continued consolidation of the defense industry in the US it was now increasingly more difficult to get a competitive price for products. At the other end you have Russia who have a largely state controlled economy which has minimal external debt. The US could potentially suffer Eurozone type problems while it is difficult to forsee Russia suffering from these problems for the forseeable future though this does come to the detriment of near term growth and capital input

- I think the key reason why the US has been able to maintain economic leadership is that the rest of the world sees them as necessary. Think about the debt loads in some of the recently beleagured Eurozone members. Technically, the US should be in trouble but an internal loop that it has created to buy it's own debt has helped to stablise things especially in light of difficulties after their banking sector troubles

- it was previously said that a lot of money was stolen during the Russia's economic problems of the late 90's (some people say the West, others say it was Russian's themselves). Cynical side of me is thinking we're dealing with former KGB/FSB staffers. They know how to hide money and apparently there are now records of money coming out of no where back into local Russian accounts. My guess is that at least some of the money may have been appropriated for 'national security/interest' purposes and have been used to buffer the local economy for precisely these type of circumstances. Even if they are technically criminals, they are nationalist criminals... Wild theory, but possible...

- in spite of what is said, outright war between the major powers is only going to start if there is a miscalcaulation and something extremely stupid occurs. Other states will take a long time before they reach the quantity/quality required to challenge abroad to genuinely make a play at the US. More likely, the US is going to de-legitimise itself or else it will do so with the aide of it's enemies over time. This will occur through a number of different ways

- I think some of the major problems with current US leadership include: allies can put up with the US spying on them but not if it takes too much of an advantage from such circumstances, decision making (particularly in the geo-political sphere) proves to be less than wise over an extended period of time, if others can continue to bridge the gap (technologically, financially, etc...) it's power will be diminished, if the US is shown to be not much more different than the rest of the rest of the world then it's perception in the eyes of others will reduce, in spite of of the amount of money that is thrown at their intelligence and defense services I'm not certain that they're getting genuine value. If they have the intelligence but make the wrong decision, if they have get thrown by mis-information, etc... then it's almost as bad as starting without any intelligence. I feel that the primary threat will not come via arms, it will come via a whole bunch of other mechanisms, such as the 'Statecraft' that was employed by the US in Ukraine, cyber-warfare, NOCs, funding/support of politically favourable opponents, etc...

UKRAINE Regime Change: Nuland Admits US Invested $5 Billion To "Assist" Democracy

- the problem with propaganda is that it's too difficult to get to the bottom of it. The end citizen just doesn't have the time to break through and figure out what the truth is. The irony is that journalist's primary end goal now is to make money in an industry which is in decline. To that end they need to push the sensationalist perspective. The one that is also likely to support the propagandist position

- as I said in my previous post, this is what I thought would happen with regards to control of oil prices...

- a lot of people believe in the position of 'Total War' to get your enemy to lay down and accept defeat. Problem is that like the notion of 'deterrence' it only works on people who are sane. Moreover, use it on people who are sane and it will likely mean that they will turn against you. There is a different between justice and vengeance

- I think the process of de-radicalisation needs to start from as young as possible. My family used to have certain opinions of other cultures. I never paid notice to them since I often had friends from those cultures and their opinions made no sense to me in the context of my own life. I believe kids of counter cultures should be encouraged to interact to interact as early as impossible to make radicalisation seem rediculous. It's also the reason why I believe in a more balanced media, political, religious, etc... response in general

Some interesting quotes I came across recently...

- Skunk — a foul-smelling liquid first sprayed on Palestinian protesters as a form of crowd control in 2008 — has become one of the characteristic scents of the Israeli occupation.


After Skunk makes contact with a person or object, the putrid stench can last for days and can cause nausea and vomiting. The smell is overpowering, similar to a skunk’s spray but worse, smelling as if it has been mixed with raw sewage, sulfur and rotting animal corpses.

“The overpowering odor of the Skunk drives rioters away — and keeps them away — effectively shutting down any escalating situation,” Odortec’s website says.


Because of its putrid smell and the confidentiality surrounding its composition — the BBC revealed yeast and baking powder are among the ingredients — rumors abound among Palestinians as to what is in Skunk.

“People say it’s made of chemicals like gas, dirty water and shit,” said Jaber Abu Rahmah, a Palestinian living in the West Bank village of Bil’in, where residents hold weekly protests against Israel’s wall on their land.

“When it hits the ground, the smell stays bad for a long time,” he said. “When it hits you, you need to take many showers to clean yourself from the dirty smell.”

Odortec did not respond to Al Jazeera’s request for comment in time for publication but maintains on its website that Skunk is organic, nontoxic and even drinkable.

- Soviet dictator Josef Stalin once said that it doesn't matter who votes, but who counts the votes.

- Propaganda and deception are as important as ground fighting, according to Gerasimov, who wrote, "Information war is now the main type of war, preparing the way for military action." The goal: reducing "the fighting potential of the enemy" and "deluding the opposite side's military and political leaders," Gerasimov wrote.

- This week Congress sent President Obama a huge defense bill of $612 billion. The Senate ok’d it and sent it on to the president, who is threatening to veto it. Secretary of Defense Ash Carter told reporters that he wants Obama to veto it.

Here’s my old rant—we need to remember this about the Defense Department:

There is a law that every Federal department be able to be audited and is audited every year. The Defense Department refuses to update its department-wide computer system so that it even CAN be audited. Therefore, it can't be audited. That means it is completely able to spend anything it wants to spend on anything it wants to spend money on. Defense Department employees, for instance, may be able to just take money out of those billions and put it in their own bank accounts if they want to and there is no audit to find that out. They can spend $5,000 on a toilet seat if they want to. They can pay their brother-in-law a million dollars a year to mop the floors and if he keeps his mouth shut, nobody might ever know about it. As an accountant I know that large companies always have an Internal Audit department AND they get audited by all manner of outside agencies. The company I retired from had a special place set up just for auditors in one of their buildings. We were audited almost around the calendar by such organizations as the IRS and regulatory agencies. We were not able to say…. Well, we don’t have the right kind of accounting system for you to audit us, so just go away and stop bugging us.

Recall just here in Kansas a woman was charged last week with stealing $400,000 from her company by taking money from the company and putting it into her own account. She's just one of the thieves that was caught. The defense department may be full of thieves, but we'll never know because THEY refuse to be audited. And they get away with that.

- Eisenhower called it a "military-industrial complex" and warned about what it could become (he remembered America when it was not a war state). Julian Assange went a step further and called it the "military-industrial-imperialist complex."

The operative word there might be "industrial," revealing the true owners of "our" military. For one thing, it is a massive corporate welfare program (something the Kochs say they don't like and want to get rid of).

"According to the Bulletin of the Atomic Scientists, every single one of the top ten weapons contractors was convicted of or admitted to defrauding the government between 1980 and 1992. For example:

* Grumman paid the government $20 million to escape criminal liability for coercing subcontractors into making political contributions.

* Lockheed was convicted of paying millions in bribes to obtain classified planning documents.

* Northrop was fined $17 million for falsifying test data on its cruise missiles and fighter jets.

* Rockwell was fined $5.5 million for committing criminal fraud against the Air Force."

And that's just the beginning of this list (and it's an old list):

- The current American "empire" consists of over 1,400 military bases in more than 120 countries. Meanwhile, Russia has only 12 military bases, one of which is in Syria that the Americans are currently trying to close down. Washington tried to do the same in Crimea, Assange said.

- The US, in conjunction with Saudi Arabia and Qatar, attempted to train and support Sunni extremists to overthrow the Assad regime. Some of those Sunni extremists ended up going crazy and declaring a Medeival caliphate putting the Pentagon and Langley in the hilarious position of being forced to classify al-Qaeda as "moderate." The situation spun out of control leading to hundreds of thousands of civilian deaths and when Washington finally decided to try and find real "moderates" to help contain the Frankenstein monster the CIA had created in ISIS (there were of course numerous other CIA efforts to arm and train anti-Assad fighters, see below for the fate of the most "successful" of those groups), the effort ended up being a complete embarrassment that culminated with the admission that only "four or five" remained and just days after that admission, those "four or five" were car jacked by al-Qaeda in what was perhaps the most under-reported piece of foreign policy comedy in history.

- The plight of 74-year-old British expatriate Karl Andree — facing 350 lashes in Saudi Arabia — shines a disturbing light on the repulsive relationship between the monstrous Gulf state and Britain’s leaders.

It is a relationship that generates billions of pounds annually in trade and arms sales for the United Kingdom.

But it comes at an enormous cost, namely our casual acceptance of the kingdom’s flagrant disregard for human rights, as well as its decades-long promotion of extremist Islam around the world.


In Saudi Arabia, public lashings and beheadings, rather than being considered outrageous and inhumane, are — apart from football matches — the only form of public entertainment. So if his punishment does go ahead, he will be surrounded by jeering locals as he is forced to lie face-down on the ground.

Then a white-robed Islamist enforcer will exact the barbaric punishment as verses from the Koran are blasted from the speakers of nearby mosques.

The lashings could go on for months, or even years, because they are given in bouts of a dozen or so at a time.

The rationale for this appalling dragging out of the punishment is that, in a surreal act of supposed mercy, Mr Andree would not receive each subsequent round until his wounds from previous lashings had healed.

With good reason, his family fear that he may die before the ordeal is completed.

Only North Korea’s justice system and human rights record is worse than that of our apparently invaluable ally Saudi Arabia, which has already beheaded almost 200 men and women this year.

Tens of thousands of political prisoners are thought to be incarcerated without trial in the kingdom — where political parties are banned, the media is government-controlled, women live in total seclusion and imported workers from impoverished countries are paid a pittance and treated little better than slaves.

- In what CNN recently called her "most iconic moment," the then-first lady delivered what's now considered a landmark speech, declaring that "women's rights are human rights" and, without naming China, blasting governments that deny women the right to plan their own families, force abortions or have women sterilized against their will.

For China, it was an affront — too direct and, officials felt, profoundly hypocritical. The speech was censored, and within days the state-backed media was on the attack, lambasting the United States for its own record on women.

- SEOUL - Prior to departing for her visit to the United States this week, South Korean President Park Geun-hye issued a controversial directive requiring schools to use history textbooks issued by the national government.

The new measure targets secondary school students and will replace current textbooks from eight different publishing companies.

Park’s conservative supporters have criticized some of these textbooks as ideologically biased, more critical of South Korea’s authoritarian past than of North Korea’s communist totalitarian regime. 

"History education should not divide the citizens and students over political strife and ideological conflicts," said President Park.


Critics say the Park government directive also seems hypocritical given that Seoul has criticized Japan’s Prime Minister Shinzo Abe for issuing government sanctioned history books accused of downplaying atrocities committed against Koreans during Japanese colonial rule.

“They are very similar. Japan’s right-wing supporters also claim the necessity of government issued textbooks,” Pak said.

The Seoul government assured the public that the new textbook will be politically neutral, balanced and objective.

- Despite his Jewish origins, Chomsky became an increasing critic of Israeli policy towards the Palestinians and of US foreign policy as a whole.

"The fact that Chomsky is Jewish is secondary," says public radio broadcaster David Barsamian. "What I think is more important is the justice of the Palestinian cause... there are few in the United States who've been as strong an ally and friend of the Palestinian people as Noam Chomsky."

- "In the United States maybe you get slammed or denounced or kept out of the press," says Chomsky. "In the old Soviet Union you could end up in the Gulag, if you're in a typical American dependency, let's say like El Salvador, you get your brains blown up... It differs from society to society... It's condemning power systems, so of course it's not approved."

"Anyone who goes against the grain in US political culture, is going to be marginalised," says Barsamian. "The truths that Chomsky articulates are very unpopular."

- In the Middle East, especially, considerations of raw power routinely trump any expectations of law. Truth here may also be counter-intuitive. On those endlessly perplexing matters concerning Palestinian statehood, it is finally time to understand that "Palestine's" true enemy in the region is not Israel, but instead a sordid mix of Islamist Arab forces. Once again, going forward, any Palestinian advances toward statehood would likely be to the longer-term tactical advantage of the Islamic State group.

- Migranyan pulls no punches in his opening essay. A strong state is a prerequisite for establishing lasting democracy, he says, by creating institutions and setting down the necessary rules that can guarantee that groups can eventually compete for power “without destroying [the state] and allowing for the emergence of chaos and anarchy.” The ultimate goal of any democracy is to ensure the prosperity and well-being of the citizens under its care. A sovereign democracy is one whose institutions have evolved as a result of domestic processes rather than being imposed by outside powers (whose motives, it might be added, may not be to promote real democratization but a weakening of state capacity).

- SAR SHAHI, Afghanistan - When the Islamic State fighters seized the Mahmand Valley, they poured pepper into the wounds of their enemies, said villagers. Then, they seared their hands in vats of boiling oil. A group of villagers was blindfolded, tortured and blown apart with explosives buried underneath them.

"They pulled out my brother's teeth before they forced him to sit on the bombs," recalled Malik Namos, a tribal elder who escaped the valley along with thousands of other villagers. "They are more vicious than the Taliban, than any group we have seen."

- "There's a huge difference in the way the Taliban was treating the people and the way Daesh is now," said Hayatullah. "I prefer the Taliban any day."

- The signs of the times are everywhere. Estonia is erecting a 2.5-meter-high metal mesh fence reinforced with barbed wire along much of its border with Russia—and backing it up with high-tech drones, sensors, radars, and cameras. Neighboring Latvia has announced plans to build fences along its eastern frontier. Poland plans to build new state-of-the art watchtowers on its border with Russia’s Kaliningrad exclave.

And, of course, Ukraine has floated plans to build a wall along its Russian frontier. A new era of containment, it appears, has begun. Russia’s neighbors, wary of polite little green men appearing to stir up new non-declared hybrid wars, are building walls and becoming vigilant.

- In a 2012 report for Chatham House, James Greene noted how Putin used “the corrupt transnational schemes that flowed seamlessly from Russia into the rest of the former Soviet space—and oozed beyond it” to extend his “shadow influence beyond Russia’s borders and develop a natural, ‘captured’ constituency.”

Toward this end, Moscow has used everything from shady energy deals, to webs of shell companies, to hot money in the City of London, to the financing of extremist political parties in Europe. Its success in doing so raises the economic cost of conflict, reduces resolve to resist Moscow, and gives Russia a ready-made lobby in Western capitals. The Kremlin has effectively weaponized globalization.

- Another point. AIM-120 is not only susceptible to jamming, but also known to have serious issues maneuvering after it makes it initial acceleration burn while retaining speed. This is the same problem USSR ran into with their medium range radar guided AA missiles, which took them decades to partially solve through the strange honeycomb rear wings on the missile.

AIM-120C and D exacerbate this problem by reducing wing surface of the missile to fit it into internal bays of F-22 and F/A-35. Those missiles are quite bad in terms of kill ratio at long range because of it, which means that problem discussed in this article is worse than it appears. Not only is modern AA jamming, pioneered mainly by Russians on their newer Sukhois and French on Rafale which has apparently the most advanced jamming system for fighter aircraft in the world. It’s completely integrated into the airframe from the start, making it extremely powerful to the point where Rafale was the only non-stealth aircraft that could operate in Libya without dedicated electronic warfare support aircraft like Prowler/Growler.

As has been often noted, there are effectively two ways to reach “missile immunity”. Stealth, which is preventing radar from properly seeing and/or locking on you through denial of meaningful return signal, or electronic jamming, which is confusing the radar attempting a lock. Russians and Europeans (and by extension Chinese who base their designs mainly on Russian and sometimes European designs) went hard for the latter approach, where US mastered the former.

Right now, it appears that both approaches have significant merit, but latter has a significant advantage in not compromising weapon load size or maneuverability.

- Intelligence agencies can send or recruit agents inside organized structures.

They can bug and intercept their communications, discover their arms and explosives caches and disrupt their plans. But they can’t penetrate into the heads of individuals and read their minds. Nor can they confiscate their weapons.

As Defense Minister Moshe Ya’alon said, repelling criticism leveled by one of his colleagues during a cabinet meeting this week, “Do you want us to collect all the kitchen knives in Palestinian houses?” What is in common between then and now is the readiness to sacrifice one’s life and to die.

- With its numerous free zones and lack of proper control that make it very easy to create shell companies, the United Arab Emirates is a major hub both for the transit of legal goods and the smuggling of illegal goods, including weapons. In an October 2014 report by the Belgium-based Groupe de Recherche et d’Information sur la Paix et la Sécurité, researcher Géraldine Franc mentions the United Arab United Emirates as “a hub for the smuggling of weapons and other military equipment to Iran” and more broadly “a major transit point for goods under U.N. sanctions.”

- According to Chinese-language media sites, Yang is participating in the Chinese government’s secretive 863 Program. The program is designed to create advanced technologies that will wean China off its dependence on difficult-to-obtain foreign technologies. Yang’s participation in the program involves improvements to the GPS/BD2 receiver and the network of continuously operating reference stations used for real-time kinematic satellite navigation systems, used to improve the precision of positioning data.

- A study done by PwC ranked the Philippines 127th among 189 economies reviewed in terms of ease of paying business taxes. Thailand is ranked 62nd and Malaysia is 32nd.

“The Philippines is just three notches above Sierra Leone and 12 notches above Sudan in the rankings. In fact, it’s even easier to pay taxes in Iraq, Iran and Afghanistan. What does that say about us?” Escudero asked.

“Where else will you find a country that taxes its people severely and then makes it hard for them to pay?” he said.


The United Arab Emirates and Qatar shared the top spot for ease of payments, requiring only four payments that could be done in 12 hours in the UAE and 41 hours in Qatar.

Saudi Arabia ranked third with three payments that could be accomplished in 64 hours.

Bolivia had the worst tax payment procedure with 42 tax payments requiring 1,025 hours.

Hong Kong ranked fourth in the PwC study, with only three tax payments, while Singapore ranked fifth, with five tax payments.

“How can we encourage investors to come and put their money here when a third of that will go to taxes that will be difficult to pay?” Escudero said.

But we need to put the spending in perspective. As a continent of 52 nations, all African nations combined still spend only about as much on their militaries as India, which ranks seventh in the world.

- The Russian finance minister, Anton Siluanov, said over-reliance on oil and gas over the last decade had been a fundamental error, leading to an overvalued currency and the slow death of other industries in a textbook case of the Dutch Disease.

"We should stop caring so much about the oil industry and leave more space for others. We have to take very tough decisions and redistribute our resources," he said.

The new $50 benchmark for oil is even lower than the Russian central bank's "extreme scenario" of $60 first prepared last year.

The new realism has forced the Kremlin to ditch a raft of budget commitments and to stop topping up the pension reserve fund. Oil and gas taxes make up half the state's revenue, and almost 70pc of Russia's exports.

- Putin and Sturgeon’s popularity is propelled by a force more powerful — at least to date — than the desire for better living standards. Especially in Russia, there is a pride in displaying courage and patriotism in the face of deprivation and aggression, seen as coming largely from the United States. In Scotland, the propaganda is more muted and the English enemy less clearly delineated, but nationalism needs a foe, and the English are it.

There is no question of which nationalism is more dangerous. An independent Scotland would reduce the UK’s authority, further weaken the EU and greatly damage the state itself. Russian nationalism on the other hand is a danger, perhaps a disaster, on a global scale, not least because its political success spawns imitators. For example, China, heading into harder times and taking the world with it, has a leader keen on promoting the “Chinese dream” — a stronger, more nationalistically inclined China.

Nationalism hasn’t gone out with the tide: it’s coming in waves.

- With broader economic diversification appearing unlikely any time soon—non-oil GDP has exhibited shockingly little change since the collapse of the Soviet Union—Arctic oil and gas development takes on added importance. International cooperation is what they need, but isolation and indecision is what they’re getting.

- The results of the survey show that Turkish people have negative opinions of many foreign powers. Fifty-eight percent see the U.S. negatively and 64 percent see Russia the same way. Forty-nine percent see the European Union negatively. But, 55 percent still want Turkey to become a member of the EU.

Turkey is a member of NATO. However, 47 percent say that Turkey should not use military force to defend a NATO ally if Russia attacked the ally. In addition, only 38 percent support the U.S.-led effort to fight the Islamic State in Syria and Iraq.

- Authentic is a word often used to describe Mr. Sanders. People say he is “authentic,” which means real, and not a fake personality.

Kyle Klondick says that Bernie Sanders lacks the same style or positive messaging of successful past presidential candidates — candidates like Ronald Reagan, Bill Clinton and Barack Obama.

“Sanders’ speaking style is very blunt, very direct and his stump speech is almost just this big criticism of the country and how we have problems with income inequality and other issues of fairness that are legitimate points to make, but I think that his speaking style is frankly kind of a downer.”

- No one is touching the NSA, the NSA won't allow it.

- Russia’s unspoken but unmistakable message is that Moscow is trying one— and perhaps the only— way of ending the conflict by means of a Lebanese-style segregation of Syria into zones controlled by rival militias. To Washington’s perennial concern in any Middle Eastern imbroglio, “Tell me how this ends,” Moscow responds: The Syrian conflict will be “resolved” on Russia’s terms, even if Mr. Assad proves dispensable to the Kremlin in the long run.

- Russia’s military and government spin machine has been working overtime to sell the intervention to the Russian public.

Much of the effort appears to be modelled on strategies used by Nato and other western governments during previous interventions in Iraq, the Balkans, and elsewhere.

Russia’s defence ministry, which was heavily criticised for a lack of transparency during Russia’s last official war, in Georgia in 2008, issues daily briefings including cock-pit camera and drone footage of strikes being carried out.

The briefings emphasise the “surgical” nature of strikes, consistently claim impressive results, and downplay reports of civilian casualties as “information attacks” by Russia’s enemies.

And they have sought to neuter western objections that Russia is mostly bombing anti-Assad rebels, but not Isil, by painting all anti-Assad forces as extremist “terrorists.” The “moderate opposition,” the argument goes, exists only in the imagination of American policy makers.

- The economic system of the U.S. — although often portrayed as ‘free-market’ — does not quite live up to that description, in many cases. A quick look at the telecom or energy industry shows that many monopolistic forces are at play, and big money oftentimes can get laws rewritten to preserve power and influence. Pressure from big business and labor groups is a major factor in why America is the only major world power without a nationalized healthcare system, and why there has been enormous growth in inequality, particularly as of late.

- The Bosnian War is an illustrative example of how a U.S. administration dramatically revised its approach to a bloody, sectarian conflict, and ultimately achieved a measure of peace. Four years into the war, the massacre at Srebrenica and a broader deterioration of conditions in the former Yugoslavia exposed the weakness of the U.S. strategy to that point, challenged U.S. credibility, and forced strategic change. Abandoning an ineffective approach, the United States adopted an integrated strategy that included arms embargos, economic sanctions and rewards, and airstrikes. The United States and its European allies coerced the warring factions into negotiations and prompted the pursuit of a new political reality.

- Vladimir Putin accused the U.S. of behaving like “Big Brother” and blackmailing world leaders, warning there was no guarantee for global security in one of his sharpest-ever swipes at Russia’s Soviet-era adversary.

Blamed by the U.S. and the European Union for fueling the conflict in eastern Ukraine, Putin said the Cold War’s “victors” are dismantling international laws and relations. The U.S. is acting like the “nouveau riche” as global leader, and today’s conflicts risk toppling the world order, he told the annual Valdai Club in the Black Sea resort of Sochi.

“The Cold War has ended,” Putin said yesterday. “But it ended without peace being achieved, without clear and transparent agreements on the new rules and standards.”


Putin said “global anarchy” will grow without clear mechanisms to resolve crises. The U.S.’s “self-appointed” leadership has brought no good for other nations and a unipolar world amounts to a dictatorship, he told the group of invited foreign and Russian academics and analysts in Sochi.

“The United States does not seek confrontation with Russia, but we cannot and will not compromise on the principles on which security in Europe and North America rests,” State Department spokeswoman Jen Psaki said in response yesterday in Washington.

- The embarrassments began in earnest in July when the group (which numbered barely more than 50 at the time) had its commander and deputy kidnapped by al-Qaeda, who had already played spoiler to another group of US fighters in 2014.

But the real punchline came last month when, in an update to Congress, Gen. Lloyd Austin, head of the U.S. Central Command and Undersecretary of Defense for Policy Christine Wormuth admitted that only “four or five” fighters from the train and equip program remained in battle.

As we quipped at the time: “So the only thing that the DoD’s estimate of the actual number of fighters currently on the ground has in common with the Pentagon’s original goal of recruiting 5,400 by the end of the year, is that both figures have a '4' and a '5' in them.”

- Postscript:   By way of background, the CIA admits that the U.S. overthrew the moderate, suit-and-tie-wearing, Democratically-elected prime minister of Iran in 1953. He was overthrown because he had nationalized Iran’s oil, which had previously been controlled by BP and other Western oil companies. As part of that action, the CIA admits that it hired Iranians to pose as Communists and stage bombings in Iran in order to turn the country against its prime minister.

If the U.S. hadn’t overthrown the moderate Iranian government, the fundamentalist Mullahs would have never taken over. (Moreover, the U.S. has had a large hand in strengthening radical Islam in the Middle East by supporting radicals to fight the Soviets and others).

- Psychologists who reportedly earned millions helping the CIA devise and implement post-9/11 interrogation techniques that critics call torture are facing a federal lawsuit on behalf of three men, one of whom died in the spy agency's custody.

- The Intercept's reporting on drones last week showed a high number of civilian casualties from U.S. airstrikes. The website did not identify the source of classified documents it published and no whistleblower has publicly claimed credit.

- Hollande, rated the most unpopular president in French polling history, played up his rustic roots and love of France's "amazing landscapes" in an unlikely interview for a popular monthly hunters' magazine due to be published on Wednesday.

"I've always lived with cows in the fields," Hollande told Le Chasseur Francais (The French Hunter), while reminiscing about his childhood growing up in rural Normandy.

- North Korea operates lucrative gambling websites that target mainly South Koreans, the National Intelligence Service here believes.

The National Intelligence Service told a National Assembly audit on Tuesday that around 1,100 North Korean computer experts operate from China, Malaysia and other Asian countries, each generating around US$20,000 in gambling proceeds on average.

That is seven times what the average North Korean worker earns in a year.

The NIS said North Korean gambling websites target South Koreans in particular. One online sports betting website generated around W4 billion in profits on the first half of this year, according to the NIS (US$1=W1,133).

The spy agency said North Korea also earns hard currency by selling online gaming items like magical weapons. They install cheat programs or hacking software and then sell on their undeserved perks to gamers.

The NIS estimates the illegal online gambling and item-selling market at W34 trillion worldwide and suspects huge amounts of that flow into North Korea.

- “Since the enemy cannot harm Islam through armies and military warfare, it tries to cause discord among Muslim countries and bring them to their knees by using its intelligence services,” Alavi said on Tuesday.

He added that the enemy spy agencies are securing the survival of the Israeli regime by sowing discord among Muslim countries.

Alavi said the enemy switched from deploying military to psychological warfare for protecting the Israeli regime after Tel Aviv’s defeats in its military invasions.

He said that such plots have failed to harm Iran thanks to the valuable guidelines of the Leader of the Islamic Revolution Ayatollah Seyyed Ali Khamenei, stressing the importance of maintaining unity and security in Iran.

“Irrespective of ethnicity and religion, the Islamic [Republic of] Iran has not hesitated to help Muslims every time an adversity has befallen them. [That is] because the sacred establishment of the Islamic Republic of Iran knows no ethnic and religious demarcation.”

- If you are a low-income person, it is, depending upon where you live, very difficult to find normal banking. Banks don’t want you. And what people are forced to do is go to payday lenders who charge outrageously high interest rates. You go to check-cashing places, which rip you off. And, yes, I think that the postal service, in fact, can play an important role in providing modest types of banking service to folks who need it.

- In Mosul alone, Islamic State may have stolen 2,300 Humvees, moving many across the border back into Syria — perhaps owing to fears of a counter-offensive. This has led to some interesting episodes whereby captured Humvees went into battle in Syria, only to be seized by Kurdish forces.

The American-led coalition has frantically tried to counter the threat from these vehicles — which Islamic State transforms into mobile and near-unstoppable suicide bombs. Thousands of air strikes and hundreds of anti-tank missiles supplied to the Iraqi army have helped abate the danger but not eliminate it.

As in Vietnam, it’s an apt reminder of just how botched many of the efforts to confront and defeat that terrorist gang have been to date.

- Speaking Tuesday at a ceremony at the Kremlin, Putin said in televised comments that the FSB intelligence agency this year had foiled 20 terror plots, arrested 560 militants and killed 112 others in Russia's North Caucasus in raids and clashes.

Putin asked the FSB to increase its efforts in preventing terror attacks as well as uncovering militants' links to international groups.

Islamic insurgency has been brewing in the North Caucasus following two wars in Chechnya in the 1990s. In neighboring Dagestan, the insurgents _ who want to carve out a state governed by their strict interpretation of Islamic law _ clash with law enforcement officers almost daily. Moscow says some of these militants have links to Islamic State.

- “The primary mission of our fighter aircraft should remain the defence of North America, not stealth first-strike capability,” the Liberal party writes. That mission includes intercepting enemy planes and ships; American and Canadian fighter jets occasionally intercept Russian bombers in international airspace near their coastlines.

Canada has planned to buy 60 F-35s to replace its 30-year-old CF-18 Hornets, but the purchase has been debated for years — memorably stirred by a 2014 video of two boys playing with toy fighter jets. When a boy says he bought an F-35 with the $10 given to him by his grandfather, his brother says he bought three Super Hornets.

Experts say the Super Hornet — cheaper and easily integrated into the Canadian Air Force — is indeed the most likely replacement for the F-35.

“What they really want is something that guarantees air sovereignty, and frankly, the CF-18 has done the job and chances are the Super Hornet will do the job,” said Richard Aboulafia, vice president for analysis at the Teal Group consulting firm.

- "I can defend the invasion of Iraq," Reich told me. "What did the invasion of Iraq do? It caused all of the people who would’ve otherwise come and attacked us and killed Americans on our soil — it caused them to go to Iraq and die there. That may sound very brutal, or whatever, but we have seen what has happened when you have an administration like the current one, that did not realize what Bush had done; sent the troops home from Iraq; created a vacuum that was filled by ISIS. And they’re killing Americans and everyone else — they’re mostly killing Muslims. I lay that at the feet of the Obama Administration."

Russell Coker: LUV Server Upgrade to Jessie

Wed, 2015-10-21 16:26

On Sunday night I started the process of upgrading the LUV server to Debian/Jessie from Debian/Wheezy. My initial plan was to just upgrade Apache first but dependencies required upgrading systemd too.

One problem I’ve encountered in the past is that the Wheezy version of systemd will often hang on an upgrade to a newer version. Generally the solution to this is to run “systemctl daemon-reexec” from another terminal. The problem in this case was that not all the libraries needed for systemd had been installed, so systemd could re-exec itself but immediately aborted. The kernel really doesn’t like it when process 1 aborts repeatedly and apparently immediately hanging is the result. At the time I didn’t know this, all I knew was that my session died and the server stopped responding to pings immediately after I requested a reexec.

The LUV server is hosted at VPAC for free. As their staff have actual work to do they couldn’t spend a lot of time working on the LUV server. They told me that the screen was flickering and suspected a VGA cable. I got to the VPAC server room with the spare LUV server (LUV had been given 3 almost identical Sun servers from Barwon Water) at 16:30. By 17:30 I had fixed the core problem (boot with “init=/bin/bash“, mount the root filesystem rw, finish the upgrade of systemd and it’s dependencies, and then reboot normally). That got it into a stage where the Xen server for Wikimedia Au was working but most LUV functionality wasn’t working.

By 23:00 on Monday I had the full list server functionality working for users, this is the main feature that users want when it’s not near a meeting time. I can’t remember whether it was Monday night or Tuesday morning when I got the Drupal site going (the main LUV web site). Last night at midnight I got the last of the Mailman administrative interface going, I admit I could have got it going a bit earlier by putting SE Linux in permissive mode, but I don’t think that the members would have benefited from that (I’ll upload a SE Linux policy package that gets Mailman working on Jessie soon).

Now it’s Wednesday and I’m still fixing some cron jobs. Along the way I noticed some problems with excessive disk space use that I’m fixing now and I’ve also removed some Wikimedia related configuration files that were obsolete and would have prevented anyone from using a address to subscribe to the LUV mailing lists.

Now I believe that everything is working correctly and generally working better than before.

Lessons Learned

While Sunday night wasn’t a bad time to start the upgrade it wasn’t the best. If I had started the upgrade on Monday morning there would have been less down-time. Another possibility might be to do the upgrade while near the VPAC office during business hours, I could have started the upgrade while at a nearby cafe and then visited the server room immediately if something went wrong.

Doing an upgrade on a day when there’s no meeting within a week was a good choice. It wasn’t really a conscious choice as I’m usually doing other LUV work near the meeting day which precludes doing other LUV work that doesn’t need to be done soon. But in future it would be best to consciously plan upgrades for a date when users aren’t going to need the service much.

While the Wheezy systemd bug is unlikely to ever be fixed there are work-arounds that shouldn’t result in a broken server. At the moment it seems that the best option would be to kill -9 the systemctl processes that hang until the packages that systemd depends on are installed. The problem is that the upgrade hangs while the new systemctl tries to tell the old systemd to restart daemons. If we can get past that to the stage where the shared objects are installed then it should be ok.

The Apache upgrade from 2.2.x to 2.4.x changed the operation of some access control directives and it took me some time to work out how to fix that. Doing a Google search on the differences between those would have led me to the Apache document about upgrading from 2.2 to 2.4 [1]. That wouldn’t have prevented some down-time of the web sites but would have allowed me to prepare for it and to more quickly fix the problems when they became apparent. Also the rather confusing configuration of the LUV server (supporting many web sites that are no longer used) didn’t help things. I think that removing cruft from an installation before an upgrade would be better than waiting until after things break.

Next time I do an upgrade of such a server I’ll write notes about it while I go. That will give a better blog post about it if it becomes newsworthy enough to be blogged about and also more opportunities to learn better ways of doing it.

Sorry for the inconvenience.

Related posts:

  1. Virgin Mobile CRM Upgrade Failure I’ve recently got a new Xperia X10 Android phone for...
  2. Debian SE Linux Status June 2012 It’s almost the Wheezy freeze time and I’ve been working...
  3. I need an LMTP server I am working on a system where a front-end mail...

David Rowe: Simulating the DMR Modem

Wed, 2015-10-21 15:30

Brady O’Brien has been doing some fine work simulating the 4FSK DMR modem, based on the waveform description in the ETSI spec. It’s not a classic non-coherent 4FSK modem design. Rather it appears designed to easily integrate with legacy analog FM modulators and demodulators.

Here is the block diagram of a regular non-coherent 2FSK demod. For 4FSK there would be 4 arms, but you get the idea:

The DMR modem uses Root Raised Cosine (RRC) filters and a FM modulator and demodulator:

Here are the performance curves produced by fsk4.m:

The best we could do with our simulation is 5-6dB poorer than the theoretical performance of non-coherent 4FSK. This made me suspect we had a bug. However this performance loss compared to theory is consistent with other FSK modems I have simulated that run through legacy analog modulators, rather than using ideal demodulators.

Have we done something wrong? Does anyone have figures for DMR modem Eb/No versus BER? Perhaps with have an error in our simulation. Perhaps the high BER is tolerable for the higher layers of DMR, given the amount of FEC they’ve got it wrapped in. Once you’re over a certain threshold, FEC will take care of it.

Our simulation is consistent with the Minimum Detectable Signal (MDS) figures given for commercial DMR radios, for example 2% BER at a MDS of -120dBm. Our curve above suggests Eb/No=11dB for BER=0.02. Plugging that into a MDS calculation, and assuming a receiver Noise Figure (NF) of 2dB, and the DMR bit rate of 9600 bit/s:

    MDS = -174 + 10log10(Rb) + Eb/No + NF

    = -174 + 10*log10(9600) + 11 + 2

    = -121 dBm

If we had an ideal modem, and Codec 2 at 1200 bit/s, we could get a MDS of -135dBm, or -132dBm with 2400 bit/s over the channel to support two-slot TDMA just like DMR. That’s a huge margin. The modem matters. A lot.

It’s been really nice to have some one else working with me on modem code – thanks Brady! He has done a great job on getting his head around modem implementation. Brady also worked out how to run Octave code on simulation on parallel cores which is a fine innovation. Until now I had been stuck on one core.

Stewart Smith: An update on using Tor on Android

Wed, 2015-10-21 10:26

Back in 2012 I wrote a blog post on using Tor on Android which has proved quite popular over the years.

These days, there is the OrFox browser, which is from The Tor Project and is likely the current best way to browse the web through Tor on your Android device.

If you’re still using the custom setup Firefox, I’d recommend giving OrFox a try – it’s been working quite well for me.

sthbrx - a POWER technical blog: A tale of two Dockers

Tue, 2015-10-20 15:25

(This was published in an internal technical journal last week, and is now being published here. If you already know what Docker is, feel free to skim the first half.)

Docker seems to be the flavour of the month in IT. Most attention is focussed on using Docker for the deployment of production services. But that’s not all Docker is good for. Let’s explore Docker, and two ways I use it as a software developer.

Docker: what is it?

Docker is essentially a set of tools to deal with containers and images.

To make up an artificial example, say you are developing a web app. You first build an image: a file system which contains the app, and some associated metadata. The app has to run on something, so you also install things like Python or Ruby and all the necessary libraries, usually by installing a minimal Ubuntu and any necessary packages.1 You then run the image inside an isolated environment called a container.

You can have multiple containers running the same image, (for example, your web app running across a fleet of servers) and the containers don’t affect each other. Why? Because Docker is designed around the concept of immutability. Containers can write to the image they are running, but the changes are specific to that container, and aren’t preserved beyond the life of the container.2 Indeed, once built, images can’t be changed at all, only rebuilt from scratch.

However, as well as enabling you to easily run multiple copies, another upshot of immutability is that if your web app allows you to upload photos, and you restart the container, your photos will be gone. Your web app needs to be designed to store all of the data outside of the container, sending it to a dedicated database or object store of some sort.

Making your application Docker friendly is significantly more work than just spinning up a virtual machine and installing stuff. So what does all this extra work get you? Three main things: isolation, control and, as mentioned, immutability.

Isolation makes containers easy to migrate and deploy, and easy to update. Once an image is built, it can be copied to another system and launched. Isolation also makes it easy to update software your app depends on: you rebuild the image with software updates, and then just deploy it. You don’t have to worry about service A relying on version X of a library while service B depends on version Y; it’s all self contained.

Immutability also helps with upgrades, especially when deploying them across multiple servers. Normally, you would upgrade your app on each server, and have to make sure that every server gets all the same sets of updates. With Docker, you don’t upgrade a running container. Instead, you rebuild your Docker image and re-deploy it, and you then know that the same version of everything is running everywhere. This immutability also guards against the situation where you have a number of different servers that are all special snowflakes with their own little tweaks, and you end up with a fractal of complexity.

Finally, Docker offers a lot of control over containers, and for a low performance penalty. Docker containers can have their CPU, memory and network controlled easily, without the overhead of a full virtual machine. This makes it an attractive solution for running untrusted executables.3

As an aside: despite the hype, very little of this is actually particularly new. Isolation and control are not new problems. All Unixes, including Linux, support ‘chroots’. The name comes from “change root”: the system call changes the processes idea of what the file system root is, making it impossible for it to access things outside of the new designated root directory. FreeBSD has jails, which are more powerful, Solaris has Zones, and AIX has WPARs. Chroots are fast and low overhead. However, they offer much lower ability to control the use of system resources. At the other end of the scale, virtual machines (which have been around since ancient IBM mainframes) offer isolation much better than Docker, but with a greater performance hit.

Similarly, immutability isn’t really new: Heroku and AWS Spot Instances are both built around the model that you get resources in a known, consistent state when you start, but in both cases your changes won’t persist. In the development world, modern CI systems like Travis CI also have this immutable or disposable model – and this was originally built on VMs. Indeed, with a little bit of extra work, both chroots and VMs can give the same immutability properties that Docker gives.

The control properties that Docker provides are largely as a result of leveraging some Linux kernel concepts, most notably something called namespaces.

What Docker does well is not something novel, but the engineering feat of bringing together fine-grained control, isolation and immutability, and – importantly – a tool-chain that is easier to use than any of the alternatives. Docker’s tool-chain eases a lot of pain points with regards to building containers: it’s vastly simpler than chroots, and easier to customise than most VM setups. Docker also has a number of engineering tricks to reduce the disk space overhead of isolation.

So, to summarise: Docker provides a toolkit for isolated, immutable, finely controlled containers to run executables and services.

Docker in development: why?

I don’t run network services at work; I do performance work. So how do I use Docker?

There are two things I do with Docker: I build PHP 5, and do performance regression testing on PHP 7. They’re good case studies of how isolation and immutability provide real benefits in development and testing, and how the Docker tool chain makes life a lot nicer that previous solutions.

PHP 5 builds

I use the isolation that Docker provides to make building PHP 5 easier. PHP 5 depends on an old version of Bison, version 2. Ubuntu and Debian long since moved to version 3. There are a few ways I could have solved this:

  • I could just install the old version directly on my system in /usr/local/, and hope everything still works and nothing else picks up Bison 2 when it needs Bison 3. Or I could install it somewhere else and remember to change my path correctly before I build PHP 5.
  • I could roll a chroot by hand. Even with tools like debootstrap and schroot, working in chroots is a painful process.
  • I could spin up a virtual machine on one of our development boxes and install the old version on that. That feels like overkill: why should I need to run an entire operating system? Why should I need to copy my source tree over the network to build it?

Docker makes it easy to have a self-contained environment that has Bison 2 built from source, and to build my latest source tree in that environment. Why is Docker so much easier?

Firstly, Docker allows me to base my container on an existing container, and there’s an online library of containers to build from.4 This means I don’t have to roll a base image with debootstrap or the RHEL/CentOS/Fedora equivalent.

Secondly, unlike a chroot build process, which ultimately is just copying files around, a docker build process includes the ability to both copy files from the host and run commands in the context of the image. This is defined in a file called a Dockerfile, and is kicked off by a single command: docker build.

So, my PHP 5 build container loads an Ubuntu Vivid base container, uses apt-get to install the compiler, tool-chain and headers required to build PHP 5, then installs old bison from source, copies in the PHP source tree, and builds it. The vast majority of this process – the installation of the compiler, headers and bison, can be cached, so they don’t have to be downloaded each time. And once the container finishes building, I have a fully built PHP interpreter ready for me to interact with.

I do, at the moment, rebuild PHP 5 from scratch each time. This is a bit sub-optimal from a performance point of view. I could alleviate this with a Docker volume, which is a way of sharing data persistently between a host and a guest, but I haven’t been sufficiently bothered by the speed yet. However, Docker volumes are also quite fiddly, leading to the development of tools like docker compose to deal with them. They also are prone to subtle and difficult to debug permission issues.

PHP 7 performance regression testing

The second thing I use docker for takes advantage of the throwaway nature of docker environments to prevent cross-contamination.

PHP 7 is the next big version of PHP, slated to be released quite soon. I care about how that runs on POWER, and I preferably want to know if it suddenly deteriorates (or improves!). I use Docker to build a container with a daily build of PHP 7, and then I run a benchmark in it. This doesn’t give me a particularly meaningful absolute number, but it allows me to track progress over time. Building it inside of Docker means that I can be sure that nothing from old runs persists into new runs, thus giving me more reliable data. However, because I do want the timing data I collect to persist, I send it out of the container over the network.

I’ve now been collecting this data for almost 4 months, and it’s plotted below, along with a 5-point moving average. The most notable feature of the graph is a the drop in benchmark time at about the middle. Sure enough, if you look at the PHP repository, you will see that a set of changes to improve PHP performance were merged on July 29: changes submitted by our very own Anton Blanchard.5

Docker pain points

Docker provides a vastly improved experience over previous solutions, but there are still a few pain points. For example:

  1. Docker was apparently written by people who had no concept that platforms other than x86 exist. This leads to major issues for cross-architectural setups. For instance, Docker identifies images by a name and a revision. For example, ubuntu is the name of an image, and 15.04 is a revision. There’s no ability to specify an architecture. So, how you do specify that you want, say, a 64-bit, little-endian PowerPC build of an image versus an x86 build? There have been a couple of approaches, both of which are pretty bad. You could name the image differently: say ubuntu_ppc64le. You can also just cheat and override the ubuntu name with an architecture specific version. Both of these break some assumptions in the Docker ecosystem and are a pain to work with.

  2. Image building is incredibly inflexible. If you have one system that requires a proxy, and one that does not, you need different Dockerfiles. As far as I can tell, there are no simple ways to hook in any changes between systems into a generic Dockerfile. This is largely by design, but it’s still really annoying when you have one system behind a firewall and one system out on the public cloud (as I do in the PHP 7 setup).

  3. Visibility into a Docker server is poor. You end up with lots of different, anonymous images and dead containers, and you end up needing scripts to clean them up. It’s not clear what Docker puts on your file system, or where, or how to interact with it.

  4. Docker is still using reasonably new technologies. This leads to occasional weird, obscure and difficult to debug issues.6

Final words

Docker provides me with a lot of useful tools in software development: both in terms of building and testing. Making use of it requires a certain amount of careful design thought, but when applied thoughtfully it can make life significantly easier.

  1. There’s some debate about how much stuff from the OS installation you should be using. You need to have key dynamic libraries available, but I would argue that you shouldn’t be running long running processes other than your application. You shouldn’t, for example, be running a SSH daemon in your container. (The one exception is that you must handle orphaned child processes appropriately: see Considerations like debugging and monitoring the health of docker containers mean that this point of view is not universally shared.

  2. Why not simply make them read only? You may be surprised at how many things break when running on a read-only file system. Things like logs and temporary files are common issues.

  3. It is, however, easier to escape a Docker container than a VM. In Docker, an untrusted executable only needs a kernel exploit to get to root on the host, whereas in a VM you need a guest-to-host vulnerability, which are much rarer.

  4. Anyone can upload an image, so this does require running untrusted code from the Internet. Sadly, this is a distinctly retrograde step when compared to the process of installing binary packages in distros, which are all signed by a distro’s private key.

  5. See

  6. I hit this last week:, although maybe that’s my fault for running systemd on my laptop.

sthbrx - a POWER technical blog: Running ppc64le_hello on real hardware

Tue, 2015-10-20 15:25

So today I saw Freestanding “Hello World” for OpenPower on Hacker News. Sadly Andrei hadn’t been able to test it on real hardware, so I set out to get it running on a real OpenPOWER box. Here’s what I did.

Firstly, clone the repo, and, as mentioned in the README, comment out mambo_write. Build it.

Grab op-build, and build a Habanero defconfig. To save yourself a fair bit of time, first edit openpower/configs/habanero_defconfig to answer n about a custom kernel source. That’ll save you hours of waiting for git.

This will build you a PNOR that will boot a linux kernel with Petitboot. This is almost what you want: you need Skiboot, Hostboot and a bunch of the POWER specific bits and bobs, but you don’t actually want the Linux boot kernel.

Then, based on op-build/openpower/package/openpower-pnor/, we look through the output of op-build for a command, something like this monstrosity:

PATH="/scratch/dja/public/op-build/output/host/bin:/scratch/dja/public/op-build/output/host/sbin:/scratch/dja/public/op-build/output/host/usr/bin:/scratch/dja/public/op-build/output/host/usr/sbin:/home/dja/bin:/home/dja/bin:/home/dja/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/opt/openpower/common/x86_64/bin" /scratch/dja/public/op-build/output/build/openpower-pnor-ed1682e10526ebd85825427fbf397361bb0e34aa/ -xml_layout_file /scratch/dja/public/op-build/output/build/openpower-pnor-ed1682e10526ebd85825427fbf397361bb0e34aa/"defaultPnorLayoutWithGoldenSide.xml" -pnor_filename /scratch/dja/public/op-build/output/host/usr/powerpc64-buildroot-linux-gnu/sysroot/pnor/"habanero.pnor" -hb_image_dir /scratch/dja/public/op-build/output/host/usr/powerpc64-buildroot-linux-gnu/sysroot/hostboot_build_images/ -scratch_dir /scratch/dja/public/op-build/output/host/usr/powerpc64-buildroot-linux-gnu/sysroot/openpower_pnor_scratch/ -outdir /scratch/dja/public/op-build/output/host/usr/powerpc64-buildroot-linux-gnu/sysroot/pnor/ -payload /scratch/dja/public/op-build/output/images/"skiboot.lid" -bootkernel /scratch/dja/public/op-build/output/images/zImage.epapr -sbe_binary_filename "venice_sbe.img.ecc" -sbec_binary_filename "centaur_sbec_pad.img.ecc" -wink_binary_filename "p8.ref_image.hdr.bin.ecc" -occ_binary_filename /scratch/dja/public/op-build/output/host/usr/powerpc64-buildroot-linux-gnu/sysroot/occ/"occ.bin" -targeting_binary_filename "HABANERO_HB.targeting.bin.ecc" -openpower_version_filename /scratch/dja/public/op-build/output/host/usr/powerpc64-buildroot-linux-gnu/sysroot/openpower_version/openpower-pnor.version.txt

Replace the -bootkernel arguement with the path to ppc64le_hello, e.g.: -bootkernel /scratch/dja/public/ppc64le_hello/ppc64le_hello

Don’t forget to move it into place!

1 mv output/host/usr/powerpc64-buildroot-linux-gnu/sysroot/pnor/habanero.pnor output/images/habanero.pnor

Then we can use skiboot’s boot test script (written by Cyril and me, coincidentally!) to flash it.

1 ppc64le_hello/skiboot/external/boot-tests/ -vp -t hab2-bmc -P <path to>/habanero.pnor

It’s not going to get into Petitboot, so just interrupt it after it powers up the box and connect with IPMI. It boots, kinda:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 [11012941323,5] INIT: Starting kernel at 0x20010000, fdt at 0x3044db68 (size 0x11cc3) Hello OPAL! _start = 0x20010000 _bss = 0x20017E28 _stack = 0x20018000 _end = 0x2001A000 KPCR = 0x20017E50 OPAL = 0x30000000 FDT = 0x3044DB68 CPU0 not found? Pick your poison: Choices: (MMU = disabled): (d) 5s delay (e) test exception (n) test nested exception (f) dump FDT (M) enable MMU (m) disable MMU (t) test MMU (u) test non-priviledged code (I) enable ints (i) disable ints (H) enable HV dec (h) disable HV dec (q) poweroff 1.42486|ERRL|Dumping errors reported prior to registration

Yes, it does wrap horribly. However, the big issue here (which you’ll have to scroll to see!) is the “CPU0 not found?”. Fortunately, we can fix this with a little patch to cpu_init in main.c to test for a PowerPC POWER8:

1 2 3 4 5 6 7 8 cpu0_node = fdt_path_offset(fdt, "/cpus/cpu@0"); if (cpu0_node < 0) { cpu0_node = fdt_path_offset(fdt, "/cpus/PowerPC,POWER8@20"); } if (cpu0_node < 0) { printk("CPU0 not found?\n"); return; }

This is definitely the wrong way to do this, but it works for now.

Now, correcting for weird wrapping, we get:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 Hello OPAL! _start = 0x20010000 _bss = 0x20017E28 _stack = 0x20018000 _end = 0x2001A000 KPCR = 0x20017E50 OPAL = 0x30000000 FDT = 0x3044DB68 Assuming default SLB size SLB size = 0x20 TB freq = 512000000 [13205442015,3] OPAL: Trying a CPU re-init with flags: 0x2 Unrecoverable exception stack top @ 0x20019EC8 HTAB (2048 ptegs, mask 0x7FF, size 0x40000) @ 0x20040000 SLB entries: 1: E 0x8000000 V 0x4000000000000400 EA 0x20040000 -> hash 0x20040 -> pteg 0x200 = RA 0x20040000 EA 0x20041000 -> hash 0x20041 -> pteg 0x208 = RA 0x20041000 EA 0x20042000 -> hash 0x20042 -> pteg 0x210 = RA 0x20042000 EA 0x20043000 -> hash 0x20043 -> pteg 0x218 = RA 0x20043000 EA 0x20044000 -> hash 0x20044 -> pteg 0x220 = RA 0x20044000 EA 0x20045000 -> hash 0x20045 -> pteg 0x228 = RA 0x20045000 EA 0x20046000 -> hash 0x20046 -> pteg 0x230 = RA 0x20046000 EA 0x20047000 -> hash 0x20047 -> pteg 0x238 = RA 0x20047000 EA 0x20048000 -> hash 0x20048 -> pteg 0x240 = RA 0x20048000 ...

The weird wrapping seems to be caused by NULLs getting printed to OPAL, but I haven’t traced what causes that.

Anyway, now it largely works! Here’s a transcript of some things it can do on real hardware.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 Choices: (MMU = disabled): (d) 5s delay (e) test exception (n) test nested exception (f) dump FDT (M) enable MMU (m) disable MMU (t) test MMU (u) test non-priviledged code (I) enable ints (i) disable ints (H) enable HV dec (h) disable HV dec (q) poweroff <press e> Testing exception handling... sc(feed) => 0xFEEDFACE Choices: (MMU = disabled): (d) 5s delay (e) test exception (n) test nested exception (f) dump FDT (M) enable MMU (m) disable MMU (t) test MMU (u) test non-priviledged code (I) enable ints (i) disable ints (H) enable HV dec (h) disable HV dec (q) poweroff <press t> EA 0xFFFFFFF000 -> hash 0xFFFFFFF -> pteg 0x3FF8 = RA 0x20010000 mapped 0xFFFFFFF000 to 0x20010000 correctly EA 0xFFFFFFF000 -> hash 0xFFFFFFF -> pteg 0x3FF8 = unmap EA 0xFFFFFFF000 -> hash 0xFFFFFFF -> pteg 0x3FF8 = RA 0x20011000 mapped 0xFFFFFFF000 to 0x20011000 incorrectly EA 0xFFFFFFF000 -> hash 0xFFFFFFF -> pteg 0x3FF8 = unmap Choices: (MMU = disabled): (d) 5s delay (e) test exception (n) test nested exception (f) dump FDT (M) enable MMU (m) disable MMU (t) test MMU (u) test non-priviledged code (I) enable ints (i) disable ints (H) enable HV dec (h) disable HV dec (q) poweroff <press u> EA 0xFFFFFFF000 -> hash 0xFFFFFFF -> pteg 0x3FF8 = RA 0x20080000 returning to user code returning to kernel code EA 0xFFFFFFF000 -> hash 0xFFFFFFF -> pteg 0x3FF8 = unmap

I also tested the other functions and they all seem to work. Running non-priviledged code with the MMU on works. Dumping the FDT and the 5s delay both worked, although they tend to stress IPMI a lot. The delay seems to correspond well with real time as well.

It does tend to error out and reboot quite often, usually on the menu screen, for reasons that are not clear to me. It usually starts with something entirely uninformative from Hostboot, like this:

1 2 1.41801|ERRL|Dumping errors reported prior to registration 2.89873|Ignoring boot flags, incorrect version 0x0

That may be easy to fix, but again I haven’t had time to trace it.

All in all, it’s very exciting to see something come out of the simulator and in to real hardware. Hopefully with the proliferation of OpenPOWER hardware, prices will fall and these sorts of systems will become increasingly accessible to people with cool low level projects like this!

sthbrx - a POWER technical blog: Petitboot Autoboot Changes

Tue, 2015-10-20 15:25

The way autoboot behaves in Petitboot has undergone some significant changes recently, so in order to ward off any angry emails lets take a quick tour of how the new system works.

Old & Busted

For some context, here is the old (or current depending on what you’re running) section of the configuration screen.

This gives you three main options: don’t autoboot, autoboot from anything, or autoboot only from a specific device. For the majority of installations this is fine, such as when you have only one default option, or know exactly which device you’ll be booting from.

A side note about default options: it is important to note that not all boot options are valid autoboot options. A boot option is only considered for auto-booting if it is marked default, eg. ‘set default’ in GRUB and ‘default’ in PXE options.

New Hotness

Below is the new autoboot configuration.

The new design allows you to specify an ordered list of autoboot options. The last two of the three buttons are self explanatory - clear the list and autoboot any device, or clear the list completely (no autoboot).

Selecting the first button, ‘Add Device’ brings up the following screen:

From here you can select any device or class of device to add to the boot order. Once added to the boot order, the order of boot options can be changed with the left and right arrow keys, and removed from the list with the minus key (‘-’).

This allows you to create additional autoboot configurations such as “Try to boot from sda2, otherwise boot from the network”, or “Give priority to PXE options from eth0, otherwise try any other netboot option”. You can retain the original behaviour by only putting one option into the list (either ‘Any Device’ or a specific device).

Presently you can add any option into the list and order them how you like - which means you can do silly things like this:


Slightly prior to the boot order changes Petitboot also received an update to its IPMI handling. IPMI ‘bootdev’ commands allow you to override the current autoboot configuration remotely, either by specifying a device type to boot (eg. PXE), or by forcing Petitboot to boot into the ‘setup’ or ‘safe’ modes. IPMI overrides are either persistent or non-persistent. A non-persistent override will disappear after a successful boot - that is, a successful boot of a boot option, not booting to Petitboot itself - whereas a persistent override will, well, persist!

If there is an IPMI override currently active, it will appear in the configuration screen with an option to manually clear it:

That sums up the recent changes to autoboot; a bit more flexibility in assigning priority, and options for more detailed autoboot order if you need it. New versions of Petitboot are backwards compatible and will recognise older saved settings, so updating your firmware won’t cause your machines to start booting things at random.

sthbrx - a POWER technical blog: Joining the CAPI project

Tue, 2015-10-20 15:25

(I wrote this blog post a couple of months ago, but it’s still quite relevant.)

Hi, I’m Daniel! I work in OzLabs, part of IBM’s Australian Development Labs. Recently, I’ve been assigned to the CAPI project, and I’ve been given the opportunity to give you an idea of what this is, and what I’ll be up to in the future!

What even is CAPI?

To help you understand CAPI, think back to the time before computers. We had a variety of machines: machines to build things, to check things, to count things, but they were all specialised — good at one and only one thing.

Specialised machines, while great at their intended task, are really expensive to develop. Not only that, it’s often impossible to change how they operate, even in very small ways.

Computer processors, on the other hand, are generalists. They are cheap. They can do a lot of things. If you can break a task down into simple steps, it’s easy to get them to do it. The trade-off is that computer processors are incredibly inefficient at everything.

Now imagine, if you will, that a specialised machine is a highly trained and experienced professional, a computer processor is a hungover university student.

Over the years, we’ve tried lots of things to make student faster. Firstly, we gave the student lots of caffeine to make them go as fast as they can. That worked for a while, but you can only give someone so much caffeine before they become unreliable. Then we tried teaming the student up with another student, so they can do two things at once. That worked, so we added more and more students. Unfortunately, lots of tasks can only be done by one person at a time, and team-work is complicated to co-ordinate. We’ve also recently noticed that some tasks come up often, so we’ve given them some tools for those specific tasks. Sadly, the tools are only useful for those specific situations.

Sometimes, what you really need is a professional.

However, there are a few difficulties in getting a professional to work with uni students. They don’t speak the same way; they don’t think the same way, and they don’t work the same way. You need to teach the uni students how to work with the professional, and vice versa.

Previously, developing this interface – this connection between a generalist processor and a specialist machine – has been particularly difficult. The interface between processors and these specialised machines – known as accelerators – has also tended to suffer from bottlenecks and inefficiencies.

This is the problem CAPI solves. CAPI provides a simpler and more optimised way to interface specialised hardware accelerators with IBM’s most recent line of processors, POWER8. It’s a common ‘language’ that the processor and the accelerator talk, that makes it much easier to build the hardware side and easier to program the software side. In our Canberra lab, we’re working primarily on the operating system side of this. We are working with some external companies who are building CAPI devices and the optimised software products which use them.

From a technical point of view, CAPI provides coherent access to system memory and processor caches, eliminating a major bottleneck in using external devices as accelerators. This is illustrated really well by the following graphic from an IBM promotional video. In the non-CAPI case, you can see there’s a lot of data (the little boxes) stalled in the PCIe subsystem, whereas with CAPI, the accelerator has direct access to the memory subsystem, which makes everything go faster.

Uses of CAPI

CAPI technology is already powering a few really cool products.

Firstly, we have an implementation of Redis that sits on top of flash storage connected over CAPI. Or, to take out the buzzwords, CAPI lets us do really, really fast NoSQL databases. There’s a video online giving more details.

Secondly, our partner Mellanox is using CAPI to make network cards that run at speeds of up to 100Gb/s.

CAPI is also part of IBM’s OpenPOWER initiative, where we’re trying to grow a community of companies around our POWER system designs. So in many ways, CAPI is both a really cool technology, and a brand new ecosystem that we’re growing here in the Canberra labs. It’s very cool to be a part of!

sthbrx - a POWER technical blog: OpenPOWER Powers Forward

Tue, 2015-10-20 15:25

I wrote this blog post late last year, it is very relevant for this blog though so I’ll repost it here.

With the launch of TYAN’s OpenPOWER reference system now is a good time to reflect on the team responsible for so much of the research, design and development behind this very first ground breaking step of OpenPOWER with their start to finish involvement of this new Power platform.

ADL Canberra have been integral to the success of this launch providing the Open Power Abstraction Layer (OPAL) firmware. OPAL breathes new life into Linux on Power finally allowing Linux to run on directly on the hardware. While OPAL harnesses the hardware, ADL Canberra significantly improved Linux to sit on top and take direct control of IBMs new Power8 processor without needing to negotiate with a hypervisor. With all the Linux expertise present at ADL Canberra it’s no wonder that a Linux based bootloader was developed to make this system work. Petitboot leverage’s all the resources of the Linux kernel to create a light, fast and yet extremely versatile bootloader. Petitboot provides a massive amount of tools for debugging and system configuration without the need to load an operating system.

TYAN have developed great and highly customisable hardware. ADL Canberra have been there since day 1 performing vital platform enablement (bringup) of this new hardware. ADL Canberra have put all the work into the entire software stack, low level work to get OPAL and Linux to talk to the new BMC chip as well as the higher level, enabling to run Linux in either endian and Linux is even now capable of virtualising KVM guests in either endian irrespective of host endian. Furthermore a subset of ADL Canberra have been key to getting the Coherent Accelerator Processor Interface (CAPI) off the ground, enabling more almost endless customisation and greater diversity within the OpenPOWER ecosystem.

ADL Canberra is the home for Linux on Power and the beginning of the OpenPOWER hardware sees much of the hard work by ADL Canberra come to fruition.

OpenSTEM: How to Help Your Child Become a Maker | MakerKids

Tue, 2015-10-20 13:30!How-To-Help-Your-Child-Become-a-Maker/dcsov/562094010cf2c3a4a7109d92

Let’s say your child is currently a classic consumer – they love watching TV, reading books, but they don’t really enjoy making things themselves. Or maybe they are making some things but it’s not really technological. We think any kind of making is awesome, but one of our favourite kinds is the kind where kids realize that they can build and influence the world around them. There’s an awesome Steve Jobs quote that I love, which says:

“When you grow up you tend to get told that the world is the way it is and you’re life is just to live your life inside the world. Try not to bash into the walls too much. Try to have a nice family life, have fun, save a little money.

That’s a very limited life. Life can be much broader once you discover one simple fact: Everything around you that you call life was made up by people that were no smarter than you. And you can change it, you can influence it…

Once you learn that, you’ll never be the same again.”

Imagine if you can figure this out as a child.

Rusty Russell: ccan/mem’s memeqzero iteration

Tue, 2015-10-20 11:28

On Thursday I was writing some code, and I wanted to test if an array was all zero.  First I checked if ccan/mem had anything, in case I missed it, then jumped on IRC to ask the author (and overall CCAN co-maintainer) David Gibson about it.

We bikeshedded around names: memallzero? memiszero? memeqz? memeqzero() won by analogy with the already-extant memeq and memeqstr. Then I asked:

rusty: dwg: now, how much time do I waste optimizing?

dwg: rusty, in the first commit, none

Exactly five minutes later I had it implemented and tested.

The Naive Approach: Times: 1/7/310/37064 Bytes: 50 bool memeqzero(const void *data, size_t length) { const unsigned char *p = data; while (length) { if (*p) return false; p++; length--; } return true; }

As a summary, I’ve give the nanoseconds for searching through 1,8,512 and 65536 bytes only.

Another 20 minutes, and I had written that benchmark, and an optimized version.

128-byte Static Buffer: Times: 6/8/48/5872 Bytes: 108

Here’s my first attempt at optimization; using a static array of 128 bytes of zeroes and assuming memcmp is well-optimized for fixed-length comparisons.  Worse for small sizes, much better for big.

const unsigned char *p = data; static unsigned long zeroes[16]; while (length > sizeof(zeroes)) { if (memcmp(zeroes, p, sizeof(zeroes))) return false; p += sizeof(zeroes); length -= sizeof(zeroes); } return memcmp(zeroes, p, length) == 0; Using a 64-bit Constant: Times: 12/12/84/6418 Bytes: 169

dwg: but blowing a cacheline (more or less) on zeroes for comparison, which isn’t necessarily a win

Using a single zero uint64_t for comparison is pretty messy:

bool memeqzero(const void *data, size_t length) {     const unsigned char *p = data;     const unsigned long zero = 0;     size_t pre;     pre = (size_t)p % sizeof(unsigned long);     if (pre) {         size_t n = sizeof(unsigned long) - pre;         if (n > length)             n = length;         if (memcmp(p, &zero, n) != 0)             return false;         p += n;         length -= n;     }     while (length > sizeof(zero)) {         if (*(unsigned long *)p != zero)             return false;         p += sizeof(zero);         length -= sizeof(zero);     }     return memcmp(&zero, p, length) == 0; }

And, worse in every way!

Using a 64-bit Constant With Open-coded Ends: Times: 4/9/68/6444 Bytes: 165

dwg: rusty, what colour is the bikeshed if you have an explicit char * loop for the pre and post?

That’s slightly better, but memcmp still wins over large distances, perhaps due to prefetching or other tricks.

Epiphany #1: We Already Have Zeroes: Times 3/5/92/5801 Bytes: 422

Then I realized that we don’t need a static buffer: we know everything we’ve already tested is zero!  So I open coded the first 16 byte compare, then memcmp()ed against the previous bytes, doubling each time.  Then a final memcmp for the tail.  Clever huh?

But it no faster than the static buffer case on the high end, and much bigger.

dwg: rusty, that is brilliant. but being brilliant isn’t enough to make things work, necessarily :p

Epiphany #2: memcmp can overlap: Times 3/5/37/2823 Bytes: 307

My doubling logic above was because my brain wasn’t completely in phase: unlike memcpy, memcmp arguments can happily overlap!  It’s still worth doing an open-coded loop to start (gcc unrolls it here with -O3), but after 16 it’s worth memcmping with the previous 16 bytes.  This is as fast as naive with as little as 2 bytes, and the fastest solution by far with larger numbers:

const unsigned char *p = data; size_t len; /* Check first 16 bytes manually */ for (len = 0; len < 16; len++) { if (!length) return true; if (*p) return false; p++; length--; } /* Now we know that's zero, memcmp with self. */ return memcmp(data, p, length) == 0;

You can find the final code in CCAN (or on Github) including the benchmark code.

Finally, after about 4 hours of random yak shaving, it turns out lightning doesn’t even want to use memeqzero() any more!  Hopefully someone else will benefit.

Stewart Smith: TianoCore (UEFI) ported to OpenPower

Tue, 2015-10-20 11:26

Recently, there’s been (actually two) ports of TianoCore (the reference implementation of UEFI firmware) to run on POWER on top of OPAL (provided by skiboot) – and it can be run in the Qemu PowerNV model.

More details: