Planet Linux Australia

Syndicate content
Planet Linux Australia - http://planet.linux.org.au
Updated: 38 min 18 sec ago

Michael Still: Percival trig

Tue, 2015-06-30 21:29
I had a pretty bad day, so I knocked off early and went for a walk before going off to the meeting at a charity I help out with. The walk was to Percival trig, which I have to say was one of the more boring trigs I've been to. Some of the forest nearly was nice enough, but the trig itself is stranded out in boring grasslands. Meh.



   



Interactive map for this route.



Tags for this post: blog pictures 20150630-percival photo canberra bushwalk

Related posts: Goodwin trig; Big Monks; Geocaching; Confessions of a middle aged orienteering marker; A quick walk through Curtin; Narrabundah trig and 16 geocaches



Comment

Michael Still: A team walk around Red Hill

Tue, 2015-06-30 09:28
My team at work is trying to get a bit more active, so a contingent from the Canberra portion of the team went for a walk around Red Hill. I managed to sneak in a side trip to Davidson trig, but it was cheating because it was from the car park at the top of the hill. A nice walk, with some cool geocaches along the way.



 



Interactive map for this route.



Tags for this post: blog pictures 20150629-davidson photo canberra bushwalk

Related posts: Goodwin trig; Big Monks; Geocaching; Confessions of a middle aged orienteering marker; A quick walk through Curtin; Narrabundah trig and 16 geocaches



Comment

Binh Nguyen: The Value of Money - Part 4

Tue, 2015-06-30 01:25
- I previously remarked that since we use the concept of 'deterrence' so readily throughout the world we are in a de-facto state of 'Cold War' whose weapons are defense, intelligence, and economics. There's a lot of interesting information out there...

http://blogs.telegraph.co.uk/news/shashankjoshi/100224247/france-should-remember-its-own-history-before-complaining-too-much-about-american-espionage/

https://wikileaks.org/gifiles/docs/11/1172615_-ct-analysis-an-economic-security-role-for-european-spy.html

http://www.wikileaks-forum.com/nsa/332/r-james-woolsey-why-we-spy-on-our-allies-17-03-2000/24575/

http://www.abc.net.au/news/2013-11-08/australian-nsa-involvement-explained/5079786

http://www.abc.net.au/news/2013-11-08/the-chinese-embassy-bugging-controversy/5079148

http://www.news.com.au/national/australia-must-choose-between-chinese-cash-and-loyalty-to-the-us-as-se-asia-tensions-rise/story-fncynjr2-1227364070887

http://rt.com/news/270529-nsa-france-economy-wikileaks/ 

http://www.bloomberg.com/news/articles/2015-06-30/why-china-wants-a-strong-euro-as-greece-teeters

http://www.smh.com.au/federal-politics/political-news/china-not-fit-for-global-leadership-says-top-canberra-official-michael-thawley-20150630-gi1o1f.html 

- it makes sense that companies try to run lean rather than try to create. Everybody knows how to save. It's much more difficult to create something of value

- advertising is a broadcast means of achieving increased transactions but in spite of targeted advertising it is still incredibly inefficient. Based on previous experience even single digit click through rates for online advertising is considered suspect/possibly fraudulent http://adage.com/article/guest-columnists/study-advertising-half-effective-previously-thought/228409/

- the easiest way of estabishing the difference between what's needed and what's wanted is to turn off all advertising around you. Once you've done that, the difference between need and want becomes very strange and the efficacy of advertising on your perspective becomes much, much clearer

- most businesses fail. A lot of people basically have trouble running a business, have flawed business models, or don't achieve enough transactions to make it worthwhile

http://www.forbes.com/sites/ericwagner/2013/09/12/five-reasons-8-out-of-10-businesses-fail/

https://www.linkedin.com/pulse/20140915223641-170128193-what-are-the-real-small-business-survival-rates

http://www.smh.com.au/business/the-economy/google-says-give-rd-tax-breaks-to-small-techies-not-big-guys-20150407-1mfy30.html

http://smallbiztrends.com/2012/09/failure-rates-by-sector-the-real-numbers.html

http://www.isbdc.org/small-business-failure-rates-causes/

http://www.washingtonpost.com/blogs/fact-checker/wp/2014/01/27/do-9-out-of-10-new-businesses-fail-as-rand-paul-claims/

- immigration is a good thing provided that the people in question bring something to the economy. I look at the Japanese situation and wonder whether or not immigration is a more cost effective means of dealing with their ageing problem than 'Abenomics'. Even if all they do is re-patriate former nationals...

http://www.koreaherald.com/view.php?ud=20150628000326

- if you run through their numbers carefully, and think about where many of the world's top companies are headed, the performance (net profit in particular) of some of them aren't any where near impressive (percentage wise) as the share price growth in recent history. There are many small/mid cap firms that would out do them (% net profit wise) if you're looking to invest

http://www.gurufocus.com/financials/AAPL&affid=45223

https://finance.yahoo.com/q/ks?s=MSFT+Key+Statistics

http://www.marketwatch.com/investing/stock/amzn/financials

http://www.marketwatch.com/investing/stock/goog/financials

https://investor.google.com/financial/tables.html

- in software engineering people continually harp on about the benefits of Agile, Extreme programming and so on. Basically, all it is maintaining regular contact between staff members to get the best out of a piece of work. Peer pressure and continual oversight also forces you to remain productive. Think about this in the real world. The larger the teams are the more difficult it is to maintain oversight particuarly if the manager in question is of a poor standard and there are no systems in place to maintain standards. There is also a problem with unfettered belief in this metholodgy. If in general, the team members are unproductive or of a poor standard this will ripple throughout your team

- GDP is a horrible measure of productivity. As I've stated previously, the difference between perceived, effective, and actual value basically diguises where true value lies. Go spend some time in other parts of the world. I guarantee that there will be a massive difference in the way you view productivity (productivity means amount of work completed per unit time not overall work)

- a good measure of a person's productivity/value is what happens if they take a day off or a have a break. Observe, the increase in workload for each other staff member and how they deal with it

- people keep on harping on about self interest as the best way of maintaining productivity and encouraging people to work hard. However, I have a huge problem with this as it is incredibly hard to differentiate between actual, effective, and perceived value sometimes. At one particular firm, we had difficulties with this as well. I was therefore tasked with writing an application to monitor things. The funny thing is that even people you think are productive tend to work in bursts. The main difference is the amount of time that trasnpires between each piece of work and the rate of work that occurs during each burst. The other thing that you should know is that even with senior members of staff and you have metrics it can be extremely difficult to justify their wage. Prepare to be surprised if you currently have poor oversight in your organisation. Lack of proper oversight breeds nepotism, lack of productivity, etc...

- you'll be shocked at what poor staff can do to your team. If the members in question is particularly bad he in effect takes a number of other staff out of the equation at the same time. Think about this. You all are recruited for highly skilled jobs but one team members is poor. If he continually has to rely on other staff then he in effects takes out another member of your team simultaneously (possibly more). Think about this when training new staff. Give them enough time to get a guage of what they'll be like but if they can't hold up their part of the deal be prepared to move them elsewhere or let go of them

- we like to focus in on large companies because they supposedly bring in a lot of business. The problem is if they have a monopoly. If they strangle the market of all value and don't put back in via taxes, employment, etc... the state in question could be in a lot of trouble down the line. If/when the company moves the economy would have evolved to see these companies as being a core component. Other surrouding will likely be poorly positioned to adapt when they leave for a place which offers better terms and/or conditions. The other problem is this, based on experience people are willing to except a lower wage to work for such firms (mostly for reasons of financial safety). There is no guarantee that you will be paid what you are worth

http://techcrunch.com/2015/06/28/policy-after-uber/

http://www.irishtimes.com/business/economy/smes-account-for-99-7-of-business-enterprises-in-republic-1.2035800

http://www.irishtimes.com/business/economy/economy-primed-for-sustained-growth-says-goldman-sachs-1.2143071

http://www.wsj.com/articles/SB10001424127887324787004578496803472834948

http://www.afr.com/technology/technology-companies/ireland-scraps-google-tech-company-tax-breaks-20141019-119m80

https://en.wikipedia.org/wiki/Double_Irish_arrangement

http://blogs.cfainstitute.org/investor/2015/06/11/solutions-to-a-misbehaving-finance-industry/

http://www.theguardian.com/commentisfree/2015/jun/28/david-cameron-is-abusing-magna-carta-in-abolishing-our-rights

http://www.theguardian.com/world/2015/mar/25/irelands-economy-starting-to-fire-all-cylinders-imf-report

http://www.irishtimes.com/business/economy/who-owes-more-money-the-irish-or-the-greeks-1.2236034

http://www.theguardian.com/us-news/2015/feb/02/barack-obama-tax-profits-president-budget-offshore

http://www.smh.com.au/business/multinationals-channel-more-money-through-hubs-in-singapore-switzerland-than-ever-before-tax-office-says-20150204-1363u5.html

http://www.smh.com.au/business/retail/jeff-kennett-tells-coles-to-pay-12m-to-suppliers-20150630-gi19wv.html 

- when and if a large company collapses or moves the problem is the number of others who rely on it for business

- people keep on saying that there are safe industries from off shoring and automation. I think they're naive or haven't spent enough time around good technologists. Good employees will try to automate or develop processes to get things done more efficiently. Virtually all industries (or vast chunks of them) can be automated fully given time (trust me on this. I like to read a lot...).

http://www.technologyreview.com/view/519241/report-suggests-nearly-half-of-us-jobs-are-vulnerable-to-computerization/

http://www.futuristspeaker.com/2012/02/2-billion-jobs-to-disappear-by-2030/

http://www.forbes.com/sites/jmaureenhenderson/2012/08/30/careers-are-dead-welcome-to-your-low-wage-temp-work-future/

http://theconversation.com/australia-must-prepare-for-massive-job-losses-due-to-automation-43321

http://www.theguardian.com/business/2015/jun/16/computers-could-replace-five-million-australian-jobs-within-two-decades

Only way to keep yourself safe is to be multi-skilled and entrepreneurial or else extremely skilled at a particular profession. Even then there's no guarantee that you'll be safe

http://time.com/3938678/obamacare-supreme-court-uber/

http://techcrunch.com/2015/06/28/policy-after-uber/

- sometimes I think people just don't get it. A small number of outliers is all it takes in order to change group behaviour. Even if we ban regulate/automation there will be those who adopt it without any misgivings much like organised crime, and use of illegal migrants, cash economy, etc... Only real way is to force a cashless society so that we can run algorithms to check for unusual behaviour and breed a more puritan society

- minimal but effective regulation helps to level out the playing field. Making it too complex creates possible avenues for loopholes to be exploited. Too simple and without enough coverage and you have the same problem

- obvious ways to make sustained, long term money include creating something that others need or want, else have the ability to be able to change perception, to be able to see changes and adapt, arbitrage, and using a broadcast structure

- personal experience and history of others with emerging markets such as Asia and Africa says that results can be extremely variable. Without on the ground knowledge and oversight you can just as easily make a substantial profit as a massive loss through fraud. There is very little you can do about this about from taking due diligence and having measures/knowledge to be able to deal with it should it actually occur

http://timesofindia.indiatimes.com/world/uk/India-UKs-3rd-largest-job-creator-in-2014/articleshow/47714406.cms

- in reality, very few have a genuine chance of making it 'big', "Americans raised at the top and bottom of the income ladder are likely to remain there themselves as adults. Forty-three percent of those who start in the bottom are stuck there as adults, and 70 percent remain below the middle quintile. Only 4 percent of adults raised in the bottom make it all the way to the top, showing that the "rags-to-riches" story is more often found in Hollywood than in reality."

http://www.forbes.com/sites/jmaureenhenderson/2012/08/30/careers-are-dead-welcome-to-your-low-wage-temp-work-future/

- use first mover advantage as quickly as you can but have defensive measures in place

http://www.news.com.au/finance/business/is-the-free-ride-over-for-uber/story-fnda1bsz-1227419310284

- investment from third parties (angel investment, venture capital, etc...) can vary drastically. More and more want a guaranteed return on investment at least though

- based on what I've experienced VC is much more difficult to get locally than in Europe or the United States. Luckily, more companies are willing to invest provided you are posting good numbers. One other thing I've discovered locally is that they are too lazy/unwilling to help even if the idea/s may be good though (though this is changing)

http://www.afr.com/business/health/pharmaceuticals/merck-ceo-ken-frazier-on-keytruda-and-why-australians-miss-out-on-new-drugs-20150628-ghyisc

- we don't want to live day by day or have creditors/shareholders to report to so seek the highest profit whenever possible

- you can select a lot of numbers and prove essentially anything in business but their are certain numbers that you simply can't ignore such as net profit/income

- pay a person with cash by the hour where he has to do the numbers versus lump sump and he will look at things very differently. That goes for any profession, even high earning ones

- growth is great but only if it can be sustained and it is genuine. If you have susbtantial variation in growth such as having a few fantastic years of growth and then a sudden drop off that is fed by massive debt you could be in a bit of trouble. You may say that you can just sell off assets. If the growth wasn't good enough then do you see a problem? Moreover, what if you don't have something that it considered worthwhile or easy to sell off? For a state/business, your credit risk suddenly shoots up and you may possibly be priced out of the market. Targeted, sustainable, growth should be the target not growth at all costs. The Chinese position towards economic management is actually making a lot more sense to me now though I'm not certain that it would work quite as easily or be accepted in other states. You may say that we'll invest during good times? The problem is that we're often not wise enough to know when and where to invest

http://www.businessinsider.com/krugman-europe-greece-2015-6

http://www.businessinsider.com/el-erian-on-how-greece-will-impact-markets-2015-6

http://www.dawn.com/news/1162195/putins-next-challenge-propping-up-russias-troubled-banks

- in many places you are seeing a rise of left wing parties. The worrying thing is that they'll lose sight of the benefits of capitalism and fall into the trap of a more puritan communism/socialist system which hasn't really worked over the long term in the past. The other thing to be concerned about is that a lot of them don't have solid policies or answers to the problems which currently face us

http://theconversation.com/postcard-from-spain-where-now-for-the-quiet-revolution-43779

http://blogs.channel4.com/paul-mason-blog/greece-referendum-euro-die/3978

- if more people could distinguish real value from perceived and effective value, needs and wants, we would have less assetts bubbles and price gouging across the board

http://www.news.com.au/finance/real-estate/bis-shrapnel-report-reveals-property-prices-to-fall/story-fncq3era-1227416605503?from=google_rss&google_editors_picks=true

https://www.ozbargain.com.au/node/104348

http://www.news.com.au/world/breaking-news/nz-govt-slammed-over-10m-ny-apartment/story-e6frfkui-1227416038766

- there will be those who say who cares about the collective. Capitalism is composed of boom and bust cycles. Here's the problem. Most companies require debt to survive. If they can't survive that bust cycle they will be part of a collective collapse in the economy. Moreover, based on information I've come across other developed countries have looked at the plans for the Eurozone and the ways of dealing with high debt and are basically using that as the blueprint for the future. Your assets can and will be raided in the event of the state or systemic entities getting into trouble

http://www.heraldsun.com.au/business/greeks-stashing-money-in-homes-as-deadline-looms-for-debt-repayments/story-fni0d2cj-1227403214181?from=google_rss&google_editors_picks=true

http://www.nytimes.com/2015/06/30/business/dealbook/the-hard-line-on-greece.html?_r=0

http://www.usatoday.com/story/news/2015/06/29/evening-news-roundup-monday/29466899/

- people say that we should get educated in order to have a high paying job but the problem is that we are increasingly siloed into specific roles. If we can't use the knowledge, the time and money we've spent on education has been for nothing. We require better organisation between educational curriculums and professional settings

http://www.financialexpress.com/article/companies/infosys-wipro-tech-mahindra-it-giants-revamp-culture-to-attract-young-talent-battle-start-ups/86718/

- even if governments are aware that there are problems that are cropping up with our version of capitalism, it's possible that there are those that may be saying that we have no choice but to keep the cycle going. It's the best of the worst

http://www.bbc.co.uk/news/world-europe-33303105

- globlisation essentially buys us more time before things come to a head (if they do). Most of the sceanarios point to organised debt forgiveness as a means of dealing with the problem. Private asset seizure is something that is being metioned everywhere. Raw commodities stored at secure locations may be your only source of safety if things look bad if you are a private citizen

http://www.washingtonpost.com/blogs/wonkblog/wp/2015/06/29/greece/

http://www.news.com.au/finance/small-business/those-selling-safes-are-cashing-in-on-greeces-financial-uncertainty/story-fn9evb64-1227422325045

http://www.news.com.au/finance/economy/what-a-grexit-would-look-like/story-e6frflo9-1227422412614

- if you want a resilient economy you need maintain a level playing field, flexible workforce, and possibly limit the size and influence of major companies in your economy

- I don't get it. Heaps of countries have adequate blocking technology to be help deal with this if they deem it illegal. Deploy it correctly and your rioting problem is over with...

http://www.arkansasonline.com/news/2015/jun/27/hollande-uber-unit-illegal-dismantle-it/

http://www.theguardian.com/technology/2015/jun/26/uber-expansion-meets-global-revolt-and-crackdown

http://timesofindia.indiatimes.com/tech/tech-news/Officials-hint-at-possible-win-for-Uber-in-Mexico-City/articleshow/47861342.cms

- as stated previously, I've come to the conclusion that a lot of financial instruments are useless. They effectively provide a means of making money under any conditions. If we remove these instruments from play then I think that it may be possible that we may return to less speculative markets that depend more on fundamentals

- anyone can create something of value. The issue is whether it is negligible versus tangible value. This will also determine your business model

- you may know that ther is a bubble but as China and local experiences have demonstrated popping it gracefully is far from easy. Moreover, by the time you figure out there's a bubble it may often too late. Too many people may have too many vested interests

http://www.reuters.com/article/2015/06/29/us-usa-puertorico-restructuring-idUSKCN0P903Q20150629

http://jamaica-gleaner.com/article/commentary/20150630/editorial-jamaica-no-greece

- theory helps but you won't figure out how market economies work without first hand experience

http://www.afr.com/news/policy/budget/big-government-flourishes-under-tony-abbott-and-joe-hockey-20150513-gh0sgrhttp://www.dailytelegraph.com.au/news/nsw/joe-blasts-welfare-rich-who-have-more-money-to-spend-than-workers/story-fni0cx12-1227357517141

http://www.smh.com.au/business/australiachina-free-trade-agreement-favours-chinese-investors-20150621-ghthjr.html

http://www.afr.com/technology/telstra-cuts-broadband-plan-fees-to-counter-rivals-20150626-ghyir7

http://www.afr.com/opinion/columnists/trophy-trade-deals-wont-change-the-imfs-dismal-outlook-20150628-ghysnn

http://www.brisbanetimes.com.au/act-news/uberx-australian-drivers-working-as-coequals-to-rideshare-tech-company-20150629-ghvjx1.html

http://www.dailytelegraph.com.au/business/breaking-news/hockey-flying-blind-on-negative-gearing/story-fnn9c0gv-1227417798217?nk=0b226f408634f8d8ba57220c3d074f55-1435471944

http://www.abc.net.au/news/2013-11-08/the-chinese-embassy-bugging-controversy/5079148

http://www.macleans.ca/news/world/why-refugees-are-fleeing-france-for-britain/

http://www.businessinsider.com.au/facebooks-shot-at-cisco-just-got-deadly-2015-3

http://www.theglobeandmail.com/globe-drive/culture/technology/gyroscopes-will-allow-bike-to-stay-upright-when-stopped/article24920123/

http://www.businessinsider.in/5-things-Elon-Musk-believed-would-change-the-future-of-humanity-in-1995/articleshow/46831594.cms

Craige McWhirter: Craige McWhirter: How To Delete a Cinder Snapshot with a Status of error or error_deleting With Ceph Block Storage

Mon, 2015-06-29 18:29

When deleting a volume snapshot in OpenStack you may sometimes get an error message stating that Cinder was unable to delete the snapshot.

There are a number of reasons why a snapshot may be reported by Ceph as unable to be deleted, however the most common reason in my experience has been that a Cinder client connection has not yet been closed, possibly because a client crashed.

If you were to look at the snapshots in Cinder, the status is usually error or error_deleting:

% cinder snapshot-list +--------------------------------------+--------------------------------------+----------------+------------------------------------------------------------------+------+ | ID | Volume ID | Status | Display Name | Size | +--------------------------------------+--------------------------------------+----------------+------------------------------------------------------------------+------+ | 07d75992-bf3f-4c9c-ab4e-efccdfc2fe02 | 3004d6e9-7934-4c95-b3ee-35a69f236e46 | error | snappy:3004d6e9-7934-4c95-b3ee-35a69f236e46:2015-06-26T14:00:02Z | 40 | | 2db84ec7-6e1a-41f8-9dc9-1dc14e6ecef0 | 3004d6e9-7934-4c95-b3ee-35a69f236e46 | error_deleting | snappy:3004d6e9-7934-4c95-b3ee-35a69f236e46:2015-05-18T00:00:01Z | 40 | | 47fbbfe8-643c-4711-a066-36f247632339 | 3004d6e9-7934-4c95-b3ee-35a69f236e46 | available | snappy:3004d6e9-7934-4c95-b3ee-35a69f236e46:2015-06-29T03:00:14Z | 40 | | 52c43ec8-e713-4f87-b329-3c681a3d31f2 | 3004d6e9-7934-4c95-b3ee-35a69f236e46 | error_deleting | snappy:3004d6e9-7934-4c95-b3ee-35a69f236e46:2015-06-24T14:00:02Z | 40 | | a595180f-d5c5-4c4b-a18c-ca56561f36cc | 3004d6e9-7934-4c95-b3ee-35a69f236e46 | error | snappy:3004d6e9-7934-4c95-b3ee-35a69f236e46:2015-06-25T14:00:02Z | 40 | +--------------------------------------+--------------------------------------+----------------+------------------------------------------------------------------+------+

When you check Ceph you may find the following snapshot list:

# rbd snap ls my.pool.cinder.block/volume-3004d6e9-7934-4c95-b3ee-35a69f236e46 SNAPID NAME SIZE 2069 snapshot-2db84ec7-6e1a-41f8-9dc9-1dc14e6ecef0 40960 MB 2526 snapshot-52c43ec8-e713-4f87-b329-3c681a3d31f2 40960 MB 2558 snapshot-47fbbfe8-643c-4711-a066-36f247632339 40960 MB

The astute will notice that there are only 3 snapshots listed in Ceph yet 5 listed in Cinder. We can immediately exclude 47fbbfe8 which is available in both Cinder and Ceph, so there's no issues there.

You will also notice that the snapshots with the status error are not in Ceph and the two with error_deleting are. My take on this is that for the status error, Cinder never received the message from Ceph stating that this had been deleted successfully. Whereas for the status error_deleting status, Cinder had been unsuccessful in offloading the request to Ceph.

Each status will need to be handled separately , I'm going to start with the error_deleting snapshots, which are still present in both Cinder and Ceph.

In MariaDB, set the status from error_deleting to available:

MariaDB [cinder]> update snapshots set status='available' where id = '2db84ec7-6e1a-41f8-9dc9-1dc14e6ecef0'; Query OK, 1 row affected (0.00 sec) Rows matched: 1 Changed: 1 Warnings: 0 MariaDB [cinder]> update snapshots set status='available' where id = '52c43ec8-e713-4f87-b329-3c681a3d31f2'; Query OK, 1 row affected (0.00 sec) Rows matched: 1 Changed: 1 Warnings: 0

Check in Cinder that the status of these snapshots has been updated successfully:

% cinder snapshot-list +--------------------------------------+--------------------------------------+----------------+------------------------------------------------------------------+------+ | ID | Volume ID | Status | Display Name | Size | +--------------------------------------+--------------------------------------+----------------+------------------------------------------------------------------+------+ | 07d75992-bf3f-4c9c-ab4e-efccdfc2fe02 | 3004d6e9-7934-4c95-b3ee-35a69f236e46 | error | tuttle:3004d6e9-7934-4c95-b3ee-35a69f236e46:2015-06-26T14:00:02Z | 40 | | 2db84ec7-6e1a-41f8-9dc9-1dc14e6ecef0 | 3004d6e9-7934-4c95-b3ee-35a69f236e46 | available | tuttle:3004d6e9-7934-4c95-b3ee-35a69f236e46:2015-05-18T00:00:01Z | 40 | | 47fbbfe8-643c-4711-a066-36f247632339 | 3004d6e9-7934-4c95-b3ee-35a69f236e46 | available | tuttle:3004d6e9-7934-4c95-b3ee-35a69f236e46:2015-06-29T03:00:14Z | 40 | | 52c43ec8-e713-4f87-b329-3c681a3d31f2 | 3004d6e9-7934-4c95-b3ee-35a69f236e46 | available | tuttle:3004d6e9-7934-4c95-b3ee-35a69f236e46:2015-06-24T14:00:02Z | 40 | | a595180f-d5c5-4c4b-a18c-ca56561f36cc | 3004d6e9-7934-4c95-b3ee-35a69f236e46 | error | tuttle:3004d6e9-7934-4c95-b3ee-35a69f236e46:2015-06-25T14:00:02Z | 40 | +--------------------------------------+--------------------------------------+----------------+------------------------------------------------------------------+------+

Delete the newly available snapshots from Cinder:

% cinder snapshot-delete 2db84ec7-6e1a-41f8-9dc9-1dc14e6ecef0 % cinder snapshot-delete 52c43ec8-e713-4f87-b329-3c681a3d31f2

Then check the results in Cinder and Ceph:

% cinder snapshot-list +--------------------------------------+--------------------------------------+----------------+------------------------------------------------------------------+------+ | ID | Volume ID | Status | Display Name | Size | +--------------------------------------+--------------------------------------+----------------+------------------------------------------------------------------+------+ | 07d75992-bf3f-4c9c-ab4e-efccdfc2fe02 | 3004d6e9-7934-4c95-b3ee-35a69f236e46 | error | tuttle:3004d6e9-7934-4c95-b3ee-35a69f236e46:2015-06-26T14:00:02Z | 40 | | 47fbbfe8-643c-4711-a066-36f247632339 | 3004d6e9-7934-4c95-b3ee-35a69f236e46 | available | tuttle:3004d6e9-7934-4c95-b3ee-35a69f236e46:2015-06-29T03:00:14Z | 40 | | a595180f-d5c5-4c4b-a18c-ca56561f36cc | 3004d6e9-7934-4c95-b3ee-35a69f236e46 | error | tuttle:3004d6e9-7934-4c95-b3ee-35a69f236e46:2015-06-25T14:00:02Z | 40 | +--------------------------------------+--------------------------------------+----------------+------------------------------------------------------------------+------+ # rbd snap ls my.pool.cinder.block/volume-3004d6e9-7934-4c95-b3ee-35a69f236e46 SNAPID NAME SIZE 2558 snapshot-47fbbfe8-643c-4711-a066-36f247632339 40960 MB

So we are done with Ceph now, as the error snapshots do not exist there. As they only exist in Cinder, we need to mark them as deleted in the Cinder database:

MariaDB [cinder]> update snapshots set status='deleted', deleted='1' where id = '07d75992-bf3f-4c9c-ab4e-efccdfc2fe02'; Query OK, 1 row affected (0.00 sec) Rows matched: 1 Changed: 1 Warnings: 0 MariaDB [cinder]> update snapshots set status='deleted', deleted='1' where id = 'a595180f-d5c5-4c4b-a18c-ca56561f36cc'; Query OK, 1 row affected (0.00 sec) Rows matched: 1 Changed: 1 Warnings: 0

Now check the status in Cinder:

% cinder snapshot-list +--------------------------------------+--------------------------------------+-----------+------------------------------------------------------------------+------+ | ID | Volume ID | Status | Display Name | Size | +--------------------------------------+--------------------------------------+-----------+------------------------------------------------------------------+------+ | 47fbbfe8-643c-4711-a066-36f247632339 | 3004d6e9-7934-4c95-b3ee-35a69f236e46 | available | tuttle:3004d6e9-7934-4c95-b3ee-35a69f236e46:2015-06-29T03:00:14Z | 40 | +--------------------------------------+--------------------------------------+-----------+------------------------------------------------------------------+------+

Now your errant Cinder snapshots have been removed.

Enjoy :-)

Sridhar Dhanapalan: Twitter posts: 2015-06-22 to 2015-06-28

Mon, 2015-06-29 00:27

Russell Coker: RAID Pain

Sun, 2015-06-28 21:26

One of my clients has a NAS device. Last week they tried to do what should have been a routine RAID operation, they added a new larger disk as a hot-spare and told the RAID array to replace one of the active disks with the hot-spare. The aim was to replace the disks one at a time to grow the array. But one of the other disks had an error during the rebuild and things fell apart.

I was called in after the NAS had been rebooted when it was refusing to recognise the RAID. The first thing that occurred to me is that maybe RAID-5 isn’t a good choice for the RAID. While it’s theoretically possible for a RAID rebuild to not fail in such a situation (the data that couldn’t be read from the disk with an error could have been regenerated from the disk that was being replaced) it seems that the RAID implementation in question couldn’t do it. As the NAS is running Linux I presume that at least older versions of Linux have the same problem. Of course if you have a RAID array that has 7 disks running RAID-6 with a hot-spare then you only get the capacity of 4 disks. But RAID-6 with no hot-spare should be at least as reliable as RAID-5 with a hot-spare.

Whenever you recover from disk problems the first thing you want to do is to make a read-only copy of the data. Then you can’t make things worse. This is a problem when you are dealing with 7 disks, fortunately they were only 3TB disks and only each had 2TB in use. So I found some space on a ZFS pool and bought a few 6TB disks which I formatted as BTRFS filesystems. For this task I only wanted filesystems that support snapshots so I could work on snapshots not on the original copy.

I expect that at some future time I will be called in when an array of 6+ disks of the largest available size fails. This will be a more difficult problem to solve as I don’t own any system that can handle so many disks.

I copied a few of the disks to a ZFS filesystem on a Dell PowerEdge T110 running kernel 3.2.68. Unfortunately that system seems to have a problem with USB, when copying from 4 disks at once each disk was reading about 10MB/s and when copying from 3 disks each disk was reading about 13MB/s. It seems that the system has an aggregate USB bandwidth of 40MB/s – slightly greater than USB 2.0 speed. This made the process take longer than expected.

One of the disks had a read error, this was presumably the cause of the original RAID failure. dd has the option conv=noerror to make it continue after a read error. This initially seemed good but the resulting file was smaller than the source partition. It seems that conv=noerror doesn’t seek the output file to maintain input and output alignment. If I had a hard drive filled with plain ASCII that MIGHT even be useful, but for a filesystem image it’s worse than useless. The only option was to repeatedly run dd with matching skip and seek options incrementing by 1K until it had passed the section with errors.

for n in /dev/loop[0-6] ; do echo $n ; mdadm –examine -v -v –scan $n|grep Events ; done

Once I had all the images I had to assemble them. The Linux Software RAID didn’t like the array because not all the devices had the same event count. The way Linux Software RAID (and probably most RAID implementations) work is that each member of the array has an event counter that is incremented when disks are added, removed, and when data is written. If there is an error then after a reboot only disks with matching event counts will be used. The above command shows the Events count for all the disks.

Fortunately different event numbers aren’t going to stop us. After assembling the array (which failed to run) I ran “mdadm -R /dev/md1” which kicked some members out. I then added them back manually and forced the array to run. Unfortunately attempts to write to the array failed (presumably due to mismatched event counts).

Now my next problem is that I can make a 10TB degraded RAID-5 array which is read-only but I can’t mount the XFS filesystem because XFS wants to replay the journal. So my next step is to buy another 2*6TB disks to make a RAID-0 array to contain an image of that XFS filesystem.

Finally backups are a really good thing…

Related posts:

  1. RAID and Bus Bandwidth As correctly pointed out by cmot [1] my previous post...
  2. Some RAID Issues I just read an interesting paper titled An Analysis of...
  3. Reliability of RAID ZDNet has an insightful article by Robin Harris predicting the...

Joshua Hesketh: git.openstack.org adventures

Sat, 2015-06-27 15:25

Over the past few months I started to notice occasional issues when cloning repositories (particularly nova) from git.openstack.org.

It would fail with something like

git clone -vvv git://git.openstack.org/openstack/nova . fatal: The remote end hung up unexpectedly fatal: early EOF fatal: index-pack failed

The problem would occur sporadically during our 3rd party CI runs causing them to fail. Initially these went somewhat ignored as rechecks on the jobs would succeed and the world would be shiny again. However, as they became more prominent the issue needed to be addressed.

When a patch merges in gerrit it is replicated out to 5 different cgit backends (git0[1-5].openstack.org). These are then balanced by two HAProxy frontends which are on a simple DNS round-robin.

+-------------------+ | git.openstack.org | | (DNS Lookup) | +--+-------------+--+ | | +--------+ +--------+ | A records | +-------------------v----+ +-----v------------------+ | git-fe01.openstack.org | | git-fe02.openstack.org | | (HAProxy frontend) | | (HAProxy frontend) | +-----------+------------+ +------------+-----------+ | | +-----+ +---+ | | +-----v------------------------------------v-----+ | +---------------------+ (source algorithm) | | | git01.openstack.org | | | | +---------------------+ | | +---| git02.openstack.org | | | | +---------------------+ | | +---| git03.openstack.org | | | | +---------------------+ | | +---| git04.openstack.org | | | | +---------------------+ | | +---| git05.openstack.org | | | | (HAProxy backend) | | | +---------------------+ | +------------------------------------------------+

Reproducing the problem was difficult. At first I was unable to reproduce locally, or even on an isolated turbo-hipster run. Since the problem appeared to be specific to our 3rd party tests (little evidence of it in 1st party runs) I started by adding extra debugging output to git.

We were originally cloning repositories via the git:// protocol. The debugging information was unfortunately limited and provided no useful diagnosis. Switching to https allowed for more CURL output (when using GIT_CURL_VERBVOSE=1 and GIT_TRACE=1) but this in itself just created noise. It actually took me a few days to remember that the servers are running arbitrary code anyway (a side effect of testing) and therefore cloning from the potentially insecure http protocol didn’t provide any further risk.

Over http we got a little more information, but still nothing that was conclusive at this point:

git clone -vvv http://git.openstack.org/openstack/nova . error: RPC failed; result=18, HTTP code = 200 fatal: The remote end hung up unexpectedly fatal: protocol error: bad pack header

After a bit it became more apparent that the problems would occur mostly during high (patch) traffic times. That is, when a lot of tests need to be queued. This lead me to think that either the network turbo-hipster was on was flaky when doing multiple git clones in parallel or the git servers were flaky. The lack of similar upstream failures lead me to initially think it was the former. In order to reproduce I decided to use Ansible to do multiple clones of repositories and see if that would uncover the problem. If needed I would have then extended this to orchestrating other parts of turbo-hipster in case the problem was systemic of something else.

Firstly I need to clone from a bunch of different servers at once to simulate the network failures more closely (rather than doing multiple clones on the one machine or from the one IP in containers for example). To simplify this I decided to learn some Ansible to launch a bunch of nodes on Rackspace (instead of doing it by hand).

Using the pyrax module I put together a crude playbook to launch a bunch of servers. There is likely much neater and better ways of doing this, but it suited my needs. The playbook takes care of placing appropriate sshkeys so I could continue to use them later.

--- - name: Create VMs hosts: localhost vars: ssh_known_hosts_command: "ssh-keyscan -H -T 10" ssh_known_hosts_file: "/root/.ssh/known_hosts" tasks: - name: Provision a set of instances local_action: module: rax name: "josh-testing-ansible" flavor: "4" image: "Ubuntu 12.04 LTS (Precise Pangolin) (PVHVM)" region: "DFW" count: "15" group: "raxhosts" wait: yes register: raxcreate - name: Add the instances we created (by public IP) to the group 'raxhosts' local_action: module: add_host hostname: "{{ item.name }}" ansible_ssh_host: "{{ item.rax_accessipv4 }}" ansible_ssh_pass: "{{ item.rax_adminpass }}" groupname: raxhosts with_items: raxcreate.success when: raxcreate.action == 'create' - name: Sleep to give time for the instances to start ssh #there is almost certainly a better way of doing this pause: seconds=30 - name: Scan the host key shell: "{{ ssh_known_hosts_command}} {{ item.rax_accessipv4 }} >> {{ ssh_known_hosts_file }}" with_items: raxcreate.success when: raxcreate.action == 'create' - name: Set up sshkeys hosts: raxhosts tasks: - name: Push root's pubkey authorized_key: user=root key="{{ lookup('file', '/root/.ssh/id_rsa.pub') }}"

From here I can use Ansible to work on those servers using the rax inventory. This allows me to address any nodes within my tenant and then log into them with the seeded sshkey.

The next step of course was to run tests. Firstly I just wanted to reproduce the issue, so in order to do that it would crudely set up an environment where it can simply clone nova multiple times.

--- - name: Prepare servers for git testing hosts: josh-testing-ansible* serial: "100%" tasks: - name: Install git apt: name=git state=present update_cache=yes - name: remove nova if it is already cloned shell: 'rm -rf nova' - name: Clone nova and monitor tcpdump hosts: josh-testing-ansible* serial: "100%" tasks: - name: Clone nova shell: "git clone http://git.openstack.org/openstack/nova"

By default Ansible runs with 5 folked processes. Meaning that Ansible would work on 5 servers at a time. We want to exercise git heavily (in the same way turbo-hipster does) so we use the –forks param to run the clone on all the servers at once. The plan was to keep launching servers until the error reared its head from the load.

To my surprise this happened with very few nodes (less than 15, but I left that as my minimum testing). To confirm I also ran the tests after launching further nodes to see it fail at 50 and 100 concurrent clones. It turned out that the more I cloned the higher the failure rate percentage was.

Now that I had the problem reproducing, it was time to do some debugging. I modified the playbook to capture tcpdump information during the clone. Initially git was cloning over IPv6 so I turned that off on the nodes to force IPv4 (just in case it was a v6 issue, but the problem did present itself on both networks). I also locked git.openstack.org to one IP rather than randomly hitting both front ends.

--- - name: Prepare servers for git testing hosts: josh-testing-ansible* serial: "100%" tasks: - name: Install git apt: name=git state=present update_cache=yes - name: remove nova if it is already cloned shell: 'rm -rf nova' - name: Clone nova and monitor tcpdump hosts: josh-testing-ansible* serial: "100%" vars: cap_file: tcpdump_{{ ansible_hostname }}_{{ ansible_date_time['epoch'] }}.cap tasks: - name: Disable ipv6 1/3 sysctl: name="net.ipv6.conf.all.disable_ipv6" value=1 sysctl_set=yes - name: Disable ipv6 2/3 sysctl: name="net.ipv6.conf.default.disable_ipv6" value=1 sysctl_set=yes - name: Disable ipv6 3/3 sysctl: name="net.ipv6.conf.lo.disable_ipv6" value=1 sysctl_set=yes - name: Restart networking service: name=networking state=restarted - name: Lock git.o.o to one host lineinfile: dest=/etc/hosts line='23.253.252.15 git.openstack.org' state=present - name: start tcpdump command: "/usr/sbin/tcpdump -i eth0 -nnvvS -w /tmp/{{ cap_file }}" async: 6000000 poll: 0 - name: Clone nova shell: "git clone http://git.openstack.org/openstack/nova" #shell: "git clone http://github.com/openstack/nova" ignore_errors: yes - name: kill tcpdump command: "/usr/bin/pkill tcpdump" - name: compress capture file command: "gzip {{ cap_file }} chdir=/tmp" - name: grab captured file fetch: src=/tmp/{{ cap_file }}.gz dest=/var/www/ flat=yes

This gave us a bunch of compressed capture files that I was then able to seek the help of my colleagues to debug (a particular thanks to Angus Lees). The results from an early run can be seen here: http://119.9.51.216/old/run1/

Gus determined that the problem was due to a RST packet coming from the source at roughly 60 seconds. This indicated it was likely we were hitting a timeout at the server or a firewall during the git-upload-pack of the clone.

The solution turned out to be rather straight forward. The git-upload-pack had simply grown too large and would timeout depending on the load on the servers. There was a timeout in apache as well as the HAProxy config for both frontend and backend responsiveness. The relative patches can be found at https://review.openstack.org/#/c/192490/ and https://review.openstack.org/#/c/192649/

While upping the timeout avoids the problem, certain projects are clearly pushing the infrastructure to its limits. As such a few changes were made by the infrastructure team (in particular James Blair) to improve git.openstack.org’s responsiveness.

Firstly git.openstack.org is now a higher performance (30GB) instance. This is a large step up from the previous (8GB) instances that were used as the frontend previously. Moving to one frontend additionally meant the HAProxy algorithm could be changed to leastconn to help balance connections better (https://review.openstack.org/#/c/193838/).

+--------------------+ | git.openstack.org | | (HAProxy frontend) | +----------+---------+ | | +------------------------v------------------------+ | +---------------------+ (leastconn algorithm) | | | git01.openstack.org | | | | +---------------------+ | | +---| git02.openstack.org | | | | +---------------------+ | | +---| git03.openstack.org | | | | +---------------------+ | | +---| git04.openstack.org | | | | +---------------------+ | | +---| git05.openstack.org | | | | (HAProxy backend) | | | +---------------------+ | +-------------------------------------------------+

All that was left was to see if things had improved. I rerun the test across 15, 30 and then 45 servers. These were all able to clone nova reliably where they had previously been failing. I then upped it to 100 servers where the cloning began to fail again.

Post-fix logs for those interested:

http://119.9.51.216/run15/

http://119.9.51.216/run30/

http://119.9.51.216/run45/

http://119.9.51.216/run100/

http://119.9.51.216/run15per100/

At this point, however, I’m basically performing a Distributed Denial of Service attack against git. As such, while the servers aren’t immune to a DDoS the problem appears to be fixed.

Binh Nguyen: The Value of Money - Part 3

Sat, 2015-06-27 03:38
The Western world generally saw the collapse of the Soviet Union as proof positive of the superiority of capitalism over communism/socialism. Most of the arguments bordered along the lines that the sheer scale of managing an economy, that it resulted in nepotism, bred corruption, stifled innovation, and that it didn't feed into the needs and wants of it's constituents were the reasons for their failure. The irony is that you can see many of the same flaws in communism and socialism that you see in capitalism now. Given the fact that more and more developed economies are getting into trouble I wonder whether this is the true way forward. The European Union, United States, Japan, and others have all recently endured serious economic difficulty and have been projected to continue to experience prolonged issues.



My belief that if capitalism and free market economics is to work into the future constraints must be placed on the size of firms relative to the size of the market/economy. Below are some reasons for my belief in this as well as some other notes regarding market economics:- I believe that one of the reasons we only favour free market economics because it limits the severity of problems if/when someone/something collapses. If a government collapses you have trouble everywhere. If a company collapses it only impacts the company and the immediate supply chain, distributers, retailers, etc...

- the other problem is most of the companies that grow to this size have no choice but to be driven by greed. Even if they pay their fair share of taxes most of them rely on debt of some sort in order to maintain a viable business. Without cash flow from the stock market, their creditors, etc... they can't continue to pay the bills. Hence, they must satisfy their own needs as well as that of their shareholders and creditors at the expense of those in the wider community. An example of this are the large retail chains that operate in many of the more developed countries. The problem is that their power can now rival that of the state. For instance, in Australia, "Almost 40 cents in every dollar we spend at the shops is now taken by a Woolworths or Wesfarmers-owned retail entity" with their interests including the "interests in groceries, fuel, liquor, gambling, office supplies, electronics, general merchandise, insurance and hardware, sparking concerns that consumers will pay more."

http://www.news.com.au/finance/money/coles-and-woolworths-receive-almost-40-per-cent-of-australian-retail-spending/story-e6frfmd9-1226043866311

If the chain collapses it's likely that hundreds of thousands of jobs will be lost in the event of administration/receivership. I'm arguing that we need to spread the risk a bit. If one part collapses it doesn't bring the whole thing crashing down around you

http://www.instantshift.com/2010/02/03/22-largest-bankruptcies-in-world-history/

- despite politician's complaints about MNCs/TNCs not contributing their fair share towards the tax base they aren't willing to make enough of an effort to change things to create those circumstances. There needs to be an understanding that without someone to buy their products and services these companies will go bankrupt. Large firms need employees and consumers as much as we need their tax revenue

- the irony is that we believe that since companies are large they are automatically successful, we should support them. Think about many of the recent large defense programs that were undertaken by large firms. As indicated previously, there's currently no incentive for them to help the state. They just want to survive and generate profits. The JSF program was deliberately structured in such a way that we've ended up with a fighter jet that isn't up to the original design spec, well and truly over the desired budgetary parameters, and way beyond the original design constraints putting the national security of many allied nations at risk

- progress within the context of market economics is often only facilitated through proper competition and regulation. At the moment, many of the largest donors towards political parties are large companies. This results in a significant distortion of the playing field and what the ultimate decision makers deem to be important issues.

http://www.nytimes.com/2015/06/25/business/obama-bolsters-his-leverage-with-trade-victory-but-at-a-cost.html?_r=0

https://en.wikipedia.org/wiki/Global_saving_glut

Think about nature of the pharmaceutical industry and electronics/IT industries. They both complain that progress (research and development) is difficult. The irony is that it's difficult to argue this if you're not making any worthwhile attempts at it. Both sectors sit atop enough savings to be able to cure much of the world's current woes but they have absolutely no incentive to bring it back on shore for it to be properly tax or to spend it

http://money.cnn.com/2015/03/20/investing/stocks-companies-record-cash-level-oil/

http://www.telegraph.co.uk/finance/11038180/Global-firms-sitting-on-7-trillion-war-chest.html

http://www.telegraph.co.uk/finance/budget/9150406/Budget-2012-UK-companies-are-sitting-on-billions-of-pounds-so-why-arent-they-spending-it.html

http://www.theguardian.com/commentisfree/2013/may/13/tax-havens-hidden-billions

Moreover, they more often than not just use their existing position to continue to exploit the market. A lot of electronics now is simply uneconomical or impossible to repair locally which means that you have to purchase new products once it has gone out of warranty and has failed due to engineered lifecycles (they are designed to fail after a particular period. If they didn't they would suffer the same fate that some car manufacturers have been complaining about. If they don't fail no one will buy new cars). My belief is that there should be tax concesssions if they are willing or they should be forced to invest into SME firms (which comprise the bulk of the economy) via secondary small capitalisation type funds (especially if the company doesn't know what to do with spare cash and it is left 'stagnant'). Ironically, returns on broad based funds in this area (longer term) more often than not exceed the growth of the company in question as well as the economy in general

- sometimes I wonder whether or not managing an economy (from a political perspective) is much the same as operating as a market analyst. You're effectively taking calculated bets on how the world will end up in the future. Is it possible that good economic managers need to be more lucky than skillful?

http://www.amazon.com/Random-Walk-Down-Wall-Street/dp/0393330338

https://en.wikipedia.org/wiki/A_Random_Walk_Down_Wall_Street

- in some cases, the nature of capitalism is such that the state has grown so large (because in general government services aren't profitable) that they are beginning to groan under the pressure that many of the more developed nations are now feeling. This is a case of both mis-management and a mis-understanding of how to use capitalism to your advantage

- one of the biggest contradictions in business is that it should all come down to the bottom line. The stupid aspect of this is that most companies have double digit turnover and continue to make the excuse that you should simply put up with whatever is thrown at you even if employee turnover is high. If workplaces were generally more civilised and conditions better then you would have a huge cost removed from your business (loss of employee, advertising, training, etc...)

- normally, when people are taught about life, we start with the small and simple examples and then we are pushed into more complex and advanced examples. The irony is that is often the opposite of the way we are taught about business. We are taught to dream big and win big or else crash and burn and learn your place in society. There is a major problem with this. In the Australian economy, SME business accounts for 96% of the economy. It is similar elsewhere. People leaving our educational institutions basically aren't equipped to be able to run make money by themselves right out of school. Help them/teach them how and you could help the overall economy as well as these students by equipping them to be able to look after their own needs reducing the burden on the social welfare system and giving them valuable employment experience that may be worthwhile later down the track. Most students are equipped to work for other people not to start their own company or operate as individuals

http://www.smartcompany.com.au/technology/information-technology/31806-number-of-businesses-in-australia-continues-to-stagnate-abs.html

- all politicians (and people in general) like to talk about the success of their country in being able to attract MNCs/TNCs to employ people locally. However, the problem is that they aren't the main employment drivers in the economy. Across most of the world's economies small businesses are the driving force ("Such firms comprise around 99% of all businesses in most economies and between half and three quarters of the value added. They also make a significant contribution to employment and are of interest to governments primarily for their potential to create more jobs."). One wonders that even with the increased business (direct and indirect) around a large firm when they exist in a country are you getting value for money (especially if you are subsidising their local existence)?

http://theconversation.com/growing-the-global-economy-through-small-to-medium-enterprise-the-g20-sme-conference-28307

- we actually do ourselves somewhat of a disservice by creating a perception that dreaming and living big is what you should want. Popular culture makes it feel like as though if you don't go to the right schools, work for the right companies, and so on you are a failure. The irony is that if every single graduate were taught about how to commercialise their their ideas while at school I believe that we would have a far more flexible, innovative, economy. Moreover, both they as well as economy in general would get a return on investment. It's no good telling people how to be enterpreneurial if they don't know how to be enterpreneurial.

- the irony of the large donor phenomenon is that SME business accounts for most of the activity within the economy...

http://www.smartcompany.com.au/technology/information-technology/31806-number-of-businesses-in-australia-continues-to-stagnate-abs.html

- as we've discussed previously on this blog the primary ways you can make money are to create something of value or by changing the perceived value of something such that people will want to buy it no matter what the disparity between perceived value versus effective value. Once upon a time I look at German prestige and performance vehicles to be the pinnacle or automative engineering. The more I've learned about them the less impressed I've become. If I told you the evidence points to them being the least reliable, the vehicles which depreciate the most (within any given time frame), most expensive to repair, the most expensive to insure and service, average safety, and that often only have comparable technology to other cars (once you cut through the marketing speak) you'd think that people would be incredibly stupid to purchase them. Yet, this trend continues...

http://usedcars.about.com/od/research/fl/10-Least-Reliable-Used-Car-Brands.htm

http://www.bbc.com/news/business-32332210

http://rac.com.au/motoring/motoring-advice/buying-a-car/running-costs

http://rac.com.au/news-community/road-safety-and-transport/safe-cars/how-safe-is-your-car/used-car-safety-ratings

Another good example is the upper echelons of professional sport and artistry (includes music, art, etc...). If anybody told you that you were paying several hundred dollars an hour to watch a group of individuals kick a ball you'd think that they were mad. The horrible part is when you realise top tier amateur competitions which are free to watch can be just as entertaining and skillful

- in reality, in the real world very very rarely are pure market forces at play and it often takes a lot of time for it to get through to them that for all the stuff/theory that you learn at school there's a lot more that you will also learn in the real world

- most industries fit into the following categories; something that you need or something that you want. By selling people a dream we can turn what you want into something you need and create employment from it

- if you want to make abnormal (excess) profits it's mainly about being able to distinguish between perceived, effective, and actual value. Once you can establish this you can exploit it. This is easier said than done though. Let's say you discovered Lionel Messi playing in the streets of Somalia versus Paris. More than likely, you'd value him much less if we were found in Somalia. Sometimes it can be pretty obvious, at other times it's not much different from predicting the future. For instance, the iPod was essentially a re-modled MP3 player with an integrated software solution/ecosystem, Coke is basically just a sweet, fizzy drink which is actually beaten by Pepsi in blind tests

- we like short term thinking because we like the notion that we can make a lot of money in a short space of time. That means that we can retire early, purchase luxury goods and services. The irony is that this feeds into a disparity between actual, perceived, and effective value which means that flawed businesses can continue to still work. The irony is that this flaw works in practice but in the long term it can results in asset bubbles. Valuation at the correct level is in collective's overall interests

- risk isn't necessarily directly related to reward if you're modelling is good. One way to reduce risk is to let others take it first. You might not make a massive name for yourself but should at least not break bank for a high risk project. This has been a common theme in the Russian and Chinese defense establishments where they have often taken significant cues from American technology

- it's becoming clearer to me that many financial instruments actually aren't required. The industry itself relies on the fact that that many will fall for the perceived notion that you can make a lot of money in a small amount of time or for little labour. However, the reality is that most will make a lot less that what is perceived to be the case. An example of this is the following. Many financial instruments are created for the express purpose of increasing risk exposure and therefore possible profits/losses. In reality, most people lose. It's like a casino where the house wins most the time. The other irony is the following, while liquidity can have a direct correlation with volatility (allows you to reach a more valid price earlier especially if many are involved in pricing), the same is also true in the opposite direction. It only takes a few minor outliers to be able to change the perception where value within the market exists

http://blogs.cfainstitute.org/investor/2015/06/11/solutions-to-a-misbehaving-finance-industry/

- may SME firms collapse within a short time frame but easy credit makes it easier for bad business models to continue to exist. The same is also true of the United States economy where uncompetitive industries were allowed to continue to exist for a long time without adequate trade barriers. If the barriers are lifted we should create circumstances where we force companies to alter their strategies earlier or force them to re-structure/collapse/declare bankruptcy. It will help to reduce the impact of when we provide credit to flawed companies which ultimately collapse

- the way we measure credit risk is often questionable. Financial institutions often turn away low income earners because they are considered a credit risk. I looked at this further and the rates that they are actually charged are diabolical. At one particular place, they were charging 10% for a one week loan, 15% for a 3 week loan, and then 25% for a month long loan. If the industry was so risky though how does it continue to exist? Most of the people who understand the problem have basically said that people who require this money simply have a hard time budgeting and managing their affairs. Essentially providing them with a lump sum component every once in a while makes them believe that they can spend freely. The irony is that the rest of society is also somewhat guilty of this. If we were paid cash and by the hour (rather than regular lump sum payments) and had to pay a component of our bills and other expenses each day we would look at our purchases very differently

http://www.news.com.au/finance/real-estate/stamp-duty-scandal-tony-abbott-under-pressure-to-scrap-our-worst-tax-amid-disastrous-poll/story-fndban6l-1227398035046?from=google_rss&google_editors_picks=true

http://www.perthnow.com.au/news/breaking-news/welfare-card-trial-sites-still-undecided/story-fnhrvfuw-1227398497797?nk=2dc00eb5accf0aef95bbb39faeb08ba0-1434358050

At the other end of the scale, there exists another paradox/contradiction. I've heard stories about people with relatively high incomes being denied credit even though their credit history was good (companies can't make money if you don't breach credit conditions every once in a while). Despite what we say about free market economics, regulatory frameworks, etc... the system is corrupt. It's just not as overt and no one likes/wants to admit it.

- despite what many may think of him, I think Vladamir Putin is actually trying to look after his country's best interests. The collapse of the Soviet Union gave rise to the oligarch. A circumstance that was facilitated by the nature of free market economics without an adequate framework (rules and regulations such as that provided by law). Essentially, the state was replaced by private enterprise where the needs of the many were placed lower on the pecking order than had the state still been in charge. I understand his perspective but I don't believe in the way he has gone about things

https://en.wikipedia.org/wiki/Revolutions_of_1989

https://en.wikipedia.org/wiki/Socialism

https://en.wikipedia.org/?title=Communism

https://en.wikipedia.org/wiki/Capitalism

http://www.msnbc.com/msnbc/pope-francis-rejects-communism-critique

http://ncronline.org/blogs/francis-chronicles/pope-francis-concern-poor-sign-gospel-not-red-flag-communism

http://www.marxist.com/kievs-contemporary-anti-communism-and-the-crimes-of-the-oligarchys-very-existence.htm

- people say that we should do more and spend more in the fight against organised crime. The stupid, ironic thing is that when society is unfair and unjust organised crime grows much stronger because it provides people with a way of making a living. In Europe, the Italian mafia has grown much stronger with the advent of the European economic difficulties and it was much the same in Japan when their asset bubble burst during the 90s

https://en.wikipedia.org/wiki/Lost_Decade_%28Japan%29

- the EU was borne of the fact that no one wanted war again in Europe. It feels like much the same with the rest of the world. We've used progress and better living conditions as an argument against going to war. However, the world has essentally ended up engaging in an effective 'Cold War'. Much of the world's spending revolves around the notion of deterrence. Namely, if I go to want to go to war with you I know that I'll suffer just as much damage (if not more)

https://en.wikipedia.org/wiki/List_of_countries_by_military_expenditures

http://www.globalissues.org/article/75/world-military-spending

There are a number of ways around this. By reaching a concensus for that countries will no longer attempt to project power outwards (defend yourself only, don't interfere with others. Highly unlikely.), invasion will no longer be part of the future landscape (other countries will come to the aide of those in trouble. Unlikely especially with the rise of terrorism.), or else collapse a economies such that countries will no longer be able to afford to spend on defense. The troubling thing is that the last scenario has actually been outlined in various US intelligence and defense reports. It's essentially war without war. If you can wreak havoc in someone's economy then they'll no longer be a problem for you. The irony is that the larger your intelligence apparatus the more likely you can engage in this style of activity. Previous leaked reports and WikiLeaks has made me highly skeptical that the average country doesn't engage in this style of activity.

http://www.theguardian.com/world/2013/aug/29/us-intelligence-spending-double-9-11-secret-budget

https://en.wikipedia.org/wiki/United_States_intelligence_budget

The irony is that you if you don't engage in these activities you may lose a significant advantage. If you do, you're sort of left to question whether or not you are the good guy in this affair

- people who haven't spent enough time in the real world only often understand the theory. Once you understand how things actually work your whole perspective changes. Let's take the housing asset/bubble that we may be going through. As stated previously, making abnormal profits is about managing the difference between perceieved, actual, and effective value. It's clear that in theory boosting supply may change things. The thing I've discovered is that in free market economics it only takes a small thing to change perception. Once the perception snow balls you're stuck with the same problem. This is the same whether it is a new home buyer or a foreign investor purchasing in the local market

http://www.smh.com.au/nsw/mike-bairds-400m-boost-for-infrastructure-fund-to-tackle-housing-affordability-crisis-20150621-ghtfr8.html

http://www.theglobeandmail.com/report-on-business/milk-surplus-forcing-canadas-dairy-industry-to-dump-supply/article25030753/

http://www.brisbanetimes.com.au/it-pro/rental-growth-slowdown-signals-residential-property-bust-on-the-way-20150626-ghxkdr

http://www.news.com.au/finance/real-estate/economists-claim-australia-in-midst-of-largest-housing-bubble-on-record/story-fncq3era-1227410053643?from=google_rss&google_editors_picks=true

- a business structure is simply a focal point of communication between business and consumer. It also affords the opportunity for a government to tax it more effectively

- by being so insistent on upskilling and education it makes low labour costs almost impossible to achieve. This makes a lot of infrastructure projects in developed countries impossible because they are economically unviable. A good example of this is 457 visas in Australia, and illegal immigration in the United States (especially from Mexico) which are often used and abused to acheive lower labour costs than otherwise would have been possible. Another example of this is the Snowy River Hydroelectricity project. It's said that hardly anyone on site knew English and that often people just learned on the job.

http://www.politico.com/story/2015/06/donald-trump-calls-jeb-bush-unhappy-119153.html?ml=ri

Another recent project put this into perspective. It was said that building a building infrastructure (tunnels, office blocks, etc...) in China, shipping it, and then assembling it here in Australia would be more cost effective then building it here alone. We need to give people a chance no matter what their education or skill level if we are to balance government budgets and to reduce the incidents of off-shoring without necessarily having to resort to often expensive anti-shoring techniques such as tarrifs, rebates, taxes, etc...

- our perception of success feels odd sometimes. If you look up the background of Rupert Murdoch, Donald Trump, and several others you'll see that thye are continually on the point of brankruptcy. Under normal circumstances anyone continually on verge of losing everything would be considered mediocre but in the business world they're considered successful because they can keep the whole thing going... Also, look at the poverty figures for the United States, Germany, United Kingdom, United Arab Emirates, Iran, and Japan. Notice the odd one out? Iran has been under sanction for a long time for their alleged nuclear research activities and yet the level of poverty in Iran is comparable to all these others.

https://en.wikipedia.org/wiki/List_of_countries_by_percentage_of_population_living_in_poverty

- the only other way to achieve lower costs in developed countries is to resort to automation and robots (else tap developing countries for lower priced components). I've looked at Australian car manufacturing plants and American and European plants for mass produced vehicles. The level of automation in American and European plants seem to be significantly higher with build quality that is comparable

http://www.kyodonews.net/news/2015/06/20/21340

http://forums.whirlpool.net.au/archive/2050953

- the perception is that we always should hire the best and brightest in order to get the job done and that we should try to do our best to make them happy. The irony is that I've worked on both sides of the fence. By hiring only the best and brightest (perceived to be. A lot of the time the best and brightest don't necessarily get hired based on what I've seen) and only settling on them we force wages up across the board and we make work more difficult for your existing workforce. It may even be more difficult to keep them happy. The other irony is that there are many wealthy global companies who can afford to hire away your best staff forcing prices up even further. Complete free trade works in favour of those who are already wealthy and makes it harder for those down the chain to make a living and to progress

- if all the best and brightest are hired by the same companies (based on personal experience) you aren't necessarily always going to get the best out of them. Companies have an increasing tendency (regulatory as well as political issues) to pigeon hole them into specific roles which doesn't allow them to realise their full and complete potential. The individual, company, as well as the collective lose out

http://blogs.cfainstitute.org/investor/2015/06/11/solutions-to-a-misbehaving-finance-industry/

- we believe in out current style of capitalism because we have a perception that it gives everyone a chance in life to be and do whatever they want. In reality, it's a lot more complicated. At it's very core I think it's very much like Winston Churchhill's opinion of the Westminster parlimentary system, "Democracy is the worst form of government, except for all the others."

http://www.goodreads.com/quotes/267224-democracy-is-the-worst-form-of-government-except-for-all

- it's clear that I believe in limited capitalism and for the most part we should try to work with those within our regions to reduce the chances of a systemic collapse. Currency manipulation, foreign investment law, tarrifs, taxation, etc... are all lawful means of changing the playing field. In fact, the exact same techniques that countries use to protect against trade sanctions can be used to guard the economic safety of citizens locally. By playing by the current rules and free trade we are essentially playing into the hands of the larger companies of the world (mostly based in the United States). It's a form of imperialism/conquest (deliberate or not) without necessarily having to engage in open warfare and with the effective ruler being the United States with these companies acting as proxies

http://www.theglobeandmail.com/report-on-business/international-business/european-business/europe-shutters-as-greek-banks-bleed-cash/article25033867/

http://rt.com/business/250497-obama-economy-china-trade/

- making or saving money can sometimes be counterintuitive. If you've ever worked in the IT industry in any sort of support role then you'll realise that no matter what level of support you operate at one of the main aims is to establish whether or not the problem occurs without your own area of oversight. If it is, you try and fix it, if not you ignore it and basically tell the other end to kindly go away because you often don't have the expertise to fix it, nor do you have the oversight to be able to. The medical and pharmaceutical industry is much the same. The irony is that this perspective can result in longer term harm than good. The United States budget is out of whack with one of the major causes being the high cost of drugs as well as short sighted perspective of medical practitioners who tend to not attempt to treat the problem till it's fixed but keep on managing it. Fix it if you can and the problem goes away, your budget is in better shape

http://ourfiniteworld.com/2011/04/08/whats-behind-united-states-budget-problems/

http://www.businessinsider.com/us-budget-deficit-2011-7

- if so many countries are so concerned about profit shifting why don't they simply make it un-economical/impossible to re-locate from now on? That way existing financial centres for such activity can adapt in the meantime while others countries can begin to regain some of their investment

- every company engages in anti-competitive behaviour. Even though (and others) Google are a supposed proponent of the 'Don't be Evil' mantra they still have shareholders to report to meaning that even if they don't want to they have to

- if too few countries make changes their companies are going to be subject to foreign takeover interest (friendly and non-friendly) if adequate measures aren't taken to protect them. Moreover, they will be at a competitive disadvantage when attemping to branch out. The only way to look after these interests is to look at the way companies are structured in order to look after the needs of both the individual and collective simultaneously

- making changes for a fairer and more equatible society isn't easy and the irony is that those who are already successful will always appeal to reduce the chances of the status quo changing. They will insist that since they 'made it' so can others. Moreover, there are always those within the political and public services who will always have differing opinions on how to acheive the same thing

http://www.smh.com.au/world/us-supreme-court-hands-obama-major-victory-on-obamacare-healthcare-reform-20150625-ghy1xq.html#content

http://www.seattletimes.com/seattle-news/a-pathological-refusal-to-see-any-shred-of-good-in-obamacare/

- people say that globalisation and free market capitalism is a guard against collapse. Someone in the system is always going to be looking for money or someone is always going to have money. The problem is that there's no incentive to do this. Moreover, it has been proven in the United States and Europe that pure private, free trade capitalism isn't necessarily going to fill the void should there be significant underlying problems. Even states and unions can not hold back the dam should the market burst. Moreover, firms have shareholders and creditors to report to. Without adequate safeguards in place the needs of the many are never going to be met by the few who are lucky enough to have survived (there is only one exception to this. If there is strong leadership/management in the private sector which I haven't seen to many instances of)

https://en.wikipedia.org/wiki/Great_Recession

http://www.afr.com/markets/commodities/energy/saudis-seen-escalating-battle-for-global-oil-market-share-20150618-ghrxws

https://en.wikipedia.org/wiki/2007%E2%80%9308_world_food_price_crisis

https://en.wikipedia.org/wiki/2000s_energy_crisis

http://www.news.com.au/national/breaking-news/govt-to-explore-social-impact-bonds/story-e6frfku9-1227416495203

http://www.news.com.au/world/breaking-news/pope-talking-drivel-catholic-economist/story-e6frfkui-1227416020721

Donna Benjamin: Try Drupal with Blackmesh

Sat, 2015-06-27 00:27
Friday, June 26, 2015 - 23:33Try Drupal

Arjen Lentz: Dutch Court orders Netherlands Government cut CO2 emissions by 25 percent by 2020 | Climate Citizen

Thu, 2015-06-25 20:25

http://takvera.blogspot.com.au/2015/06/dutch-court-orders-netherlands.html

A Dutch court in a landmark legal case has just handed down a verdict that the Netherlands Government has the legal duty to take measures against #climate change. Further, the court ordered that a 25% reduction of CO2 emissions, based on 1990 levels, must be accomplished by 2020 by the Dutch government in accordance with IPCC scientific recommendations for industrial countries.

[…]

Sue Higginson, Principal Solicitor for the Environmental Defenders Office (EDO) NSW, said that the same legal arguments are unlikely to be used in Australia, “Dutch civil laws are much more specific in their terms than Australian laws.” she said.

[…]

With Australia, such a case would be much less straightforward as we do not have the incorporation of international human rights or general duty of care directly in our constitution or legal framework.

Rusty Russell: Hashing Speed: SHA256 vs Murmur3

Thu, 2015-06-25 18:29

So I did some IBLT research (as posted to bitcoin-dev ) and I lazily used SHA256 to create both the temporary 48-bit txids, and from them to create a 16-bit index offset.  Each node has to produce these for every bitcoin transaction ID it knows about (ie. its entire mempool), which is normally less than 10,000 transactions, but we’d better plan for 1M given the coming blopockalypse.

For txid48, we hash an 8 byte seed with the 32-byte txid; I ignored the 8 byte seed for the moment, and measured various implementations of SHA256 hashing 32 bytes on on my Intel Core i3-5010U CPU @ 2.10GHz laptop (though note we’d be hashing 8 extra bytes for IBLT): (implementation in CCAN)

  1. Bitcoin’s SHA256: 527.7+/-0.9 nsec
  2. Optimizing the block ending on bitcoin’s SHA256: 500.4+/-0.66 nsec
  3. Intel’s asm rorx: 314.1+/-0.3 nsec
  4. Intel’s asm SSE4 337.5+/-0.5 nsec
  5. Intel’s asm RORx-x8ms 458.6+/-2.2 nsec
  6. Intel’s asm AVX 336.1+/-0.3 nsec

So, if you have 1M transactions in your mempool, expect it to take about 0.62 seconds of hashing to calculate the IBLT.  This is too slow (though it’s fairly trivially parallelizable).  However, we just need a universal hash, not a cryptographic one, so I benchmarked murmur3_x64_128:

  1. Murmur3-128: 23 nsec

That’s more like 0.046 seconds of hashing, which seems like enough of a win to add a new hash to the mix.

Jeremy Kerr: Toolchains for OpenPower petitboot environments

Thu, 2015-06-25 14:27

Since we're using buildroot for the OpenPower firmware build infrastructure, it's relatively straightforward to generate a standalone toolchain to build add-ons to the petitboot environment. This toolchain will allow you to cross-compile from your build host to an OpenPower host running the petitboot environment.

This is just a matter of using op-build's toolchain target, and specifying the destination directory in the BR2_HOST_DIR variable. For this example, we'll install into /opt/openpower/ :

sudo mkdir /opt/openpower/ sudo chown $USER /opt/openpower/ op-build BR2_HOST_DIR=/opt/openpower/ toolchain

After the build completes, you'll end up with a toolchain based in /opt/openpower.

Using the toolchain

If you add /opt/openpower/usr/bin/ to your PATH, you'll have the toolchain binaries available.

[jk@pecola ~]$ export PATH=/opt/openpower/usr/bin/:$PATH [jk@pecola ~]$ powerpc64le-buildroot-linux-gnu-gcc --version powerpc64le-buildroot-linux-gnu-gcc (Buildroot 2014.08-git-g80a2f83) 4.9.0 Copyright (C) 2014 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Currently, this toolchain isn't relocatable, so you'll need to keep it in the original directory for tools to correctly locate other toolchain components.

OpenPower doesn't (yet) specify an ABI for the petitboot environment, so there are no guarantees that a petitboot plugin will be forwards- or backwards- compatible with other petitboot environments.

Because of this, if you use this toolchain to build binaries for a petitboot plugin, you'll need to either:

  • ensure that your op-build version matches the one used for the target petitboot image; or
  • provide all necessary libraries and dependencies in your distributed plugin archive.

We're working to address this though, by defining the ABI that will be regarded as stable across petitboot builds. Stay tuned for updates.

Using the toolchain for subsequent op-build runs

Because op-build has a facility to use an external toolchain, you can re-use the toolchain build above for subsequent op-build invocations, where you want to build actual firmware binaries. If you're using multiple op-build trees, or are regularly building from scratch, this can save a lot of time as you don't need to continually rebuild the toolchain from source.

This is a matter of configuring your op-build tree to use an "External Toolchain", in the "Toolchain" screen of the menuconfig interface:

You'll need to set the toolchain path to the path you used for BR2_HOST_DIR above, with /usr appended. The other toolchain configuration parameters (kernel header series, libc type, features enabled) will need to match the parameters that were given in the initial toolchain build. However, the buildroot code will check that these match and print a helpful error message if there are any inconsistencies.

For the example toolchain built above, these are the full configuration parameters I used:

BR2_TOOLCHAIN=y BR2_TOOLCHAIN_USES_GLIBC=y BR2_TOOLCHAIN_EXTERNAL=y BR2_TOOLCHAIN_EXTERNAL_CUSTOM=y BR2_TOOLCHAIN_EXTERNAL_PREINSTALLED=y BR2_TOOLCHAIN_EXTERNAL_PATH="/opt/openpower/usr/" BR2_TOOLCHAIN_EXTERNAL_CUSTOM_PREFIX="$(ARCH)-linux" BR2_TOOLCHAIN_EXTERNAL_PREFIX="$(ARCH)-linux" BR2_TOOLCHAIN_EXTERNAL_GLIBC=y BR2_TOOLCHAIN_EXTERNAL_HEADERS_3_15=y BR2_TOOLCHAIN_EXTERNAL_CUSTOM_GLIBC=y BR2_TOOLCHAIN_EXTERNAL_INET_RPC=y BR2_TOOLCHAIN_EXTERNAL_CXX=y BR2_TOOLCHAIN_EXTRA_EXTERNAL_LIBS="" BR2_TOOLCHAIN_HAS_NATIVE_RPC=y BR2_TOOLCHAIN_HAS_THREADS=y BR2_TOOLCHAIN_HAS_THREADS_DEBUG=y BR2_TOOLCHAIN_HAS_THREADS_NPTL=y BR2_TOOLCHAIN_HAS_SHADOW_PASSWORDS=y BR2_TOOLCHAIN_HAS_SSP=y

Once that's done, anything you build using that op-build configuration will refer to the external toolchain, and use that for the general build process.

Richard Jones: PyCon Australia 2015 Programme Released

Wed, 2015-06-24 19:26

PyCon Australia is proud to release our programme for 2015, spread over the weekend of August 1st and 2nd, following our Miniconfs on Friday 31 July.

Following our largest ever response to our Call for Proposals, we are able to present two keynotes, forty eight talks and two tutorials. The conference will feature four full tracks of presentations, covering all aspects of the Python ecosystem, presented by experts and core developers of key Python technology. Our presenters cover a broad range of backgrounds, including industry, research, government and academia.

We are still finalising our Miniconf timetable, but we expect another thirty talks for Friday. We’d like to highlight the inaugural running of the Education Miniconf whose primary aim is to bring educators and the Python community closer together.

The full schedule for PyCon Australia 2015 can be found at http://2015.pycon-au.org/programme/about

PyCon Australia has endeavoured to keep tickets as affordable as possible. We are able to do so, thanks to our Sponsors and Contributors. Registrations for PyCon Australia 2015 are now open, with prices starting at AU$50 for students, and tickets for the general public starting at AU$240. All prices include GST, and more information can be found at http://2015.pycon-au.org/register/prices

We have also worked out favourable deals with accommodation providers for PyCon delegates. Find out more about the options at http://2015.pycon-au.org/register/accommodation

To begin the registration process, and find out more about each level of ticket, visit http://2015.pycon-au.org/register/prices

Important Dates to Help You Plan

June 29: Financial Assistance program closes.

July 8: Last day to Order PyCon Australia 2015 T-shirts

July 19: Last day to Advise Special Dietary Requirements

July 31 : PyCon Australia 2015 Begins

About PyCon Australia

PyCon Australia is the national conference for the Python Programming Community. The sixth PyCon Australia will be held on July 31 through August 4th, 2015 in Brisbane, bringing together professional, student and enthusiast developers with a love for developing with Python. PyCon Australia informs the country’s Python developers with presentations, tutorials and panel sessions by experts and core developers of Python, as well as the libraries and frameworks that they rely on.

To find out more about PyCon Australia 2015, visit our website at http://pycon-au.org or e-mail us at contact@pycon-au.org.

PyCon Australia is presented by Linux Australia (www.linux.org.au) and acknowledges the support of our Platinum Sponsors, Red Hat Asia-Pacific, and Netbox Blue; and our Gold sponsors, The Australian Signals Directorate and Google Australia. For full details of our sponsors, see our website.

Clinton Roy: clintonroy

Wed, 2015-06-24 14:28

PyCon Australia is proud to release our programme for 2015, spread over the weekend of August 1st and 2nd, following our Miniconfs on Friday 31 July.

Following our largest ever response to our Call for Proposals, we are able to present two keynotes, forty eight talks and two tutorials. The conference will feature four full tracks of presentations, covering all aspects of the Python ecosystem, presented by experts and core developers of key Python technology. Our presenters cover a broad range of backgrounds, including industry, research, government and academia.

We are still finalising our Miniconf timetable, but we expect another thirty talks for Friday. We’d like to highlight the inaugural running of the Education Miniconf whose primary aim is to bring educators and the Python community closer together.

The full schedule for PyCon Australia 2015 can be found at http://2015.pycon-au.org/programme/about

PyCon Australia has endeavoured to keep tickets as affordable as possible. We are able to do so, thanks to our Sponsors and Contributors. Registrations for PyCon Australia 2015 are now open, with prices starting at AU$50 for students, and tickets for the general public starting at AU$240. All prices include GST, and more information can be found at http://2015.pycon-au.org/register/prices

We have also worked out favourable deals with accommodation providers for PyCon delegates. Find out more about the options at http://2015.pycon-au.org/register/accommodation

To begin the registration process, and find out more about each level of ticket, visit http://2015.pycon-au.org/register/prices

Important Dates to Help You Plan

June 29: Financial Assistance program closes.

July 8: Last day to Order PyCon Australia 2015 T-shirts

July 19: Last day to Advise Special Dietary Requirements

July 31 : PyCon Australia 2015 Begins

About PyCon Australia

PyCon Australia is the national conference for the Python Programming Community. The sixth PyCon Australia will be held on July 31 through August 4th, 2015 in Brisbane, bringing together professional, student and enthusiast developers with a love for developing with Python. PyCon Australia informs the country’s Python developers with presentations, tutorials and panel sessions by experts and core developers of Python, as well as the libraries and frameworks that they rely on.

To find out more about PyCon Australia 2015, visit our website at http://pycon-au.org or e-mail us at contact@pycon-au.org.

PyCon Australia is presented by Linux Australia (www.linux.org.au) and acknowledges the support of our Platinum Sponsors, Red Hat Asia-Pacific, and Netbox Blue; and our Gold sponsors, The Australian Signals Directorate and Google Australia. For full details of our sponsors, see our website.



Filed under: Uncategorized

Jeremy Kerr: Custom kernels in OpenPower firmware

Wed, 2015-06-24 12:27

As of commit 2aff5ba6 in the op-build tree, we're able to easily replace the kernel in an OpenPower firmware image.

This commit adds a new partition (called BOOTKERNEL) to the PNOR image, which provides the petitboot bootloader environment. Since it's now in its own partition, we can replace the image with a custom build. Here's a little guide to doing that, using an example of using a separate branch of op-build that provides a little-endian kernel.

You can check if your currently-running firmware has this BOOTKERNEL partition by running pflash -i on the BMC. It should list BOOTKERNEL in the partition table listing:

# pflash -i Flash info: ----------- Name = Micron N25Qx512Ax Total size = 64MB Erase granule = 4KB Partitions: ----------- ID=00 part 00000000..00001000 (actual=00001000) ID=01 HBEL 00008000..0002c000 (actual=00024000) [...] ID=11 HBRT 00949000..00ca9000 (actual=00360000) ID=12 PAYLOAD 00ca9000..00da9000 (actual=00100000) ID=13 BOOTKERNEL 00da9000..01ca9000 (actual=00f00000) ID=14 ATTR_TMP 01ca9000..01cb1000 (actual=00008000) ID=15 ATTR_PERM 01cb1000..01cb9000 (actual=00008000) [...] #

If your partition table does not contain a BOOTKERNEL partition, you'll need to upgrade to a more recent PNOR image to proceed.

First (if you don't have one already), grab a suitable version of op-build. In this example, we'll use my le branch, which has little-endian support:

git clone --recursive git://github.com/jk-ozlabs/op-build.git cd op-build git checkout -b le origin/le git submodule update

Then, prepare our environment and configure for the relevant platform - in this case, habanero:

. op-build-env op-build habanero_defconfig

If you'd like to change any of the kernel config (for example, to add or remove drivers), you can do that now, using the 'linux-menuconfig' target. This is only necessary if you wish to make changes. Otherwise, the default kernel config will work.

op-build linux-menuconfig

Next, we build just the userspace and kernel parts of the firmware image, by specifying the linux26-rebuild-with-initramfs build target:

op-build linux26-rebuild-with-initramfs

If you're using a fresh op-build tree, this will take a little while, as it downloads and builds a toolchain, userspace and kernel. Once that's complete, you'll have a built kernel image in the output tree:

output/build/images/zImage.epapr

Transfer this file to the BMC, and flash using pflash. We specify the -P <PARTITION> argument to write to a single PNOR partition:

pflash -P BOOTKERNEL -e -p /tmp/zImage.epapr

And that's it! The next boot will use your newly-build kernel in the petitboot bootloader environment.

Out-of-tree kernel builds

If you'd like to replace the kernel from op-build with one from your own external source tree, you have two options. Either point op-build at your own tree, or build you own kernel using the initramfs that op-build has produced.

For the former, you can override certain op-build variables to reference a separate source. For example, to use an external git tree:

op-build LINUX_SITE=git://github.com/jk-ozlabs/linux LINUX_VERSION=v3.19

See Customising OpenPower firmware for other examples of using external sources in op-build.

The latter option involves doing a completely out-of-op-build build of a kernel, but referencing the initramfs created by op-build (which is in output/images/rootfs.cpio.xz). From your kernel source directory, add CONFIG_INITRAMFS_SOURCE argument, specifying the relevant initramfs. For example:

make O=obj ARCH=powerpc \ CONFIG_INITRAMFS_SOURCE=../op-build/output/images/rootfs.cpio.xz

Russell Coker: Smart Phones Should Measure Charge Speed

Wed, 2015-06-24 12:26

My first mobile phone lasted for days between charges. I never really found out how long it’s battery would last because there was no way that I could use it to deplete the charge in any time that I could spend awake. Even if I had managed to run the battery out the phone was designed to accept 4*AA batteries (it’s rechargeable battery pack was exactly that size) so I could buy spare batteries at any store.

Modern phones are quite different in physical phone design (phones that weigh less than 4*AA batteries aren’t uncommon), functionality (fast CPUs and big screens suck power), and use (games really drain your phone battery). This requires much more effective chargers, when some phones are intensively used (EG playing an action game with Wifi enabled) they can’t be charged as they use more power than the plug-pack supplies. I’ve previously blogged some calculations about resistance and thickness of wires for phone chargers [1], it’s obvious that there are some technical limitations to phone charging based on the decision to use a long cable at ~5V.

My calculations about phone charge rate were based on the theoretical resistance of wires based on their estimated cross-sectional area. One problem with such analysis is that it’s difficult to determine how thick the insulation is without destroying the wire. Another problem is that after repeated use of a charging cable some conductors break due to excessive bending. This can significantly increase the resistance and therefore increase the charging time. Recently a charging cable that used to be really good suddenly became almost useless. My Galaxy Note 2 would claim that it was being charged even though the reported level of charge in the battery was not increasing, it seems that the cable only supplied enough power to keep the phone running not enough to actually charge the battery.

I recently bought a USB current measurement device which is really useful. I have used it to diagnose power supplies and USB cables that didn’t work correctly. But one significant way in which it fails is in the case of problems with the USB connector. Sometimes a cable performs differently when connected via the USB current measurement device.

The CurrentWidget program [2] on my Galaxy Note 2 told me that all of the dedicated USB chargers (the 12V one in my car and all the mains powered ones) supply 1698mA (including the ones rated at 1A) while a PC USB port supplies ~400mA. I don’t think that the Note 2 measurement is particularly reliable. On my Galaxy Note 3 it always says 0mA, I guess that feature isn’t implemented. An old Galaxy S3 reports 999mA of charging even when the USB current measurement device says ~500mA. It seems to me that method the CurrentWidget uses to get the current isn’t accurate if it even works at all.

Android 5 on the Nexus 4/5 phones will tell the amount of time until the phone is charged in some situations (on the Nexus 4 and Nexus 5 that I used for testing it didn’t always display it and I don’t know why). This is an useful but it’s still not good enough.

I think that what we need is to have the phone measure the current that’s being supplied and report it to the user. Then when a phone charges slowly because apps are using some power that won’t be mistaken for a phone charging slowly due to a defective cable or connector.

Related posts:

  1. Cooling Phones According to the bureau of meteorology today is 39C. But...
  2. Dual SIM Phones vs Amaysim vs Contract for Mobile Phones Currently Dick Smith is offering two dual-SIM mobile phones for...
  3. Qi Phone Charging I have just bought a wireless phone charging system based...

Russell Coker: One Android Phone Per Child

Tue, 2015-06-23 12:26

I was asked for advice on whether children should have access to smart phones, it’s an issue that many people are discussing and seems worthy of a blog post.

Claimed Problems with Smart Phones

The first thing that I think people should read is this XKCD post with quotes about the demise of letter writing from 99+ years ago [1]. Given the lack of evidence cited by people who oppose phone use I think we should consider to what extent the current concerns about smart phone use are just reactions to changes in society. I’ve done some web searching for reasons that people give for opposing smart phone use by kids and addressed the issues below.

Some people claim that children shouldn’t get a phone when they are so young that it will just be a toy. That’s interesting given the dramatic increase in the amount of money spent on toys for children in recent times. It’s particularly interesting when parents buy game consoles for their children but refuse mobile phone “toys” (I know someone who did this). I think this is more of a social issue regarding what is a suitable toy than any real objection to phones used as toys. Obviously the educational potential of a mobile phone is much greater than that of a game console.

It’s often claimed that kids should spend their time reading books instead of using phones. When visiting libraries I’ve observed kids using phones to store lists of books that they want to read, this seems to discredit that theory. Also some libraries have Android and iOS apps for searching their catalogs. There are a variety of apps for reading eBooks, some of which have access to many free books but I don’t expect many people to read novels on a phone.

Cyber-bullying is the subject of a lot of anxiety in the media. At least with cyber-bullying there’s an electronic trail, anyone who suspects that their child is being cyber-bullied can check that while old-fashioned bullying is more difficult to track down. Also while cyber-bullying can happen faster on smart phones the victim can also be harassed on a PC. I don’t think that waiting to use a PC and learn what nasty thing people are saying about you is going to be much better than getting an instant notification on a smart phone. It seems to me that the main disadvantage of smart phones in regard to cyber-bullying is that it’s easier for a child to participate in bullying if they have such a device. As most parents don’t seem concerned that their child might be a bully (unfortunately many parents think it’s a good thing) this doesn’t seem like a logical objection.

Fear of missing out (FOMO) is claimed to be a problem, apparently if a child has a phone then they will want to take it to bed with them and that would be a bad thing. But parents could have a policy about when phones may be used and insist that a phone not be taken into the bedroom. If it’s impossible for a child to own a phone without taking it to bed then the parents are probably dealing with other problems. I’m not convinced that a phone in bed is necessarily a bad thing anyway, a phone can be used as an alarm clock and instant-message notifications can be turned off at night. When I was young I used to wait until my parents were asleep before getting out of bed to use my PC, so if smart-phones were available when I was young it wouldn’t have changed my night-time computer use.

Some people complain that kids might use phones to play games too much or talk to their friends too much. What do people expect kids to do? In recent times the fear of abduction has led to children doing playing outside a lot less, it used to be that 6yos would play with other kids in their street and 9yos would be allowed to walk to the local park. Now people aren’t allowing 14yo kids walk to the nearest park alone. Playing games and socialising with other kids has to be done over the Internet because kids aren’t often allowed out of the house. Play and socialising are important learning experiences that have to happen online if they can’t happen offline.

Apps can be expensive. But it’s optional to sign up for a credit card with the Google Play store and the range of free apps is really good. Also the default configuration of the app store is to require a password entry before every purchase. Finally it is possible to give kids pre-paid credit cards and let them pay for their own stuff, such pre-paid cards are sold at Australian post offices and I’m sure that most first-world countries have similar facilities.

Electronic communication is claimed to be somehow different and lesser than old-fashioned communication. I presume that people made the same claims about the telephone when it first became popular. The only real difference between email and posted letters is that email tends to be shorter because the reply time is smaller, you can reply to any questions in the same day not wait a week for a response so it makes sense to expect questions rather than covering all possibilities in the first email. If it’s a good thing to have longer forms of communication then a smart phone with a big screen would be a better option than a “feature phone”, and if face to face communication is preferred then a smart phone with video-call access would be the way to go (better even than old fashioned telephony).

Real Problems with Smart Phones

The majority opinion among everyone who matters (parents, teachers, and police) seems to be that crime at school isn’t important. Many crimes that would result in jail sentences if committed by adults receive either no punishment or something trivial (such as lunchtime detention) if committed by school kids. Introducing items that are both intrinsically valuable and which have personal value due to the data storage into a typical school environment is probably going to increase the amount of crime. The best options to deal with this problem are to prevent kids from taking phones to school or to home-school kids. Fixing the crime problem at typical schools isn’t a viable option.

Bills can potentially be unexpectedly large due to kids’ inability to restrain their usage and telcos deliberately making their plans tricky to profit from excess usage fees. The solution is to only use pre-paid plans, fortunately many companies offer good deals for pre-paid use. In Australia Aldi sells pre-paid credit in $15 increments that lasts a year [2]. So it’s possible to pay $15 per year for a child’s phone use, have them use Wifi for data access and pay from their own money if they make excessive calls. For older kids who need data access when they aren’t at home or near their parents there are other pre-paid phone companies that offer good deals, I’ve previously compared prices of telcos in Australia, some of those telcos should do [3].

It’s expensive to buy phones. The solution to this is to not buy new phones for kids, give them an old phone that was used by an older relative or buy an old phone on ebay. Also let kids petition wealthy relatives for a phone as a birthday present. If grandparents want to buy the latest smart-phone for a 7yo then there’s no reason to stop them IMHO (this isn’t a hypothetical situation).

Kids can be irresponsible and lose or break their phone. But the way kids learn to act responsibly is by practice. If they break a good phone and get a lesser phone as a replacement or have to keep using a broken phone then it’s a learning experience. A friend’s son head-butted his phone and cracked the screen – he used it for 6 months after that, I think he learned from that experience. I think that kids should learn to be responsible with a phone several years before they are allowed to get a “learner’s permit” to drive a car on public roads, which means that they should have their own phone when they are 12.

I’ve seen an article about a school finding that tablets didn’t work as well as laptops which was touted as news. Laptops or desktop PCs obviously work best for typing. Tablets are for situations where a laptop isn’t convenient and when the usage involves mostly reading/watching, I’ve seen school kids using tablets on excursions which seems like a good use of them. Phones are even less suited to writing than tablets. This isn’t a problem for phone use, you just need to use the right device for each task.

Phones vs Tablets

Some people think that a tablet is somehow different from a phone. I’ve just read an article by a parent who proudly described their policy of buying “feature phones” for their children and tablets for them to do homework etc. Really a phone is just a smaller tablet, once you have decided to buy a tablet the choice to buy a smart phone is just about whether you want a smaller version of what you have already got.

The iPad doesn’t appear to be able to make phone calls (but it supports many different VOIP and video-conferencing apps) so that could technically be described as a difference. AFAIK all Android tablets that support 3G networking also support making and receiving phone calls if you have a SIM installed. It is awkward to use a tablet to make phone calls but most usage of a modern phone is as an ultra portable computer not as a telephone.

The phone vs tablet issue doesn’t seem to be about the capabilities of the device. It’s about how portable the device should be and the image of the device. I think that if a tablet is good then a more portable computing device can only be better (at least when you need greater portability).

Recently I’ve been carrying a 10″ tablet around a lot for work, sometimes a tablet will do for emergency work when a phone is too small and a laptop is too heavy. Even though tablets are thin and light it’s still inconvenient to carry, the issue of size and weight is a greater problem for kids. 7″ tablets are a lot smaller and lighter, but that’s getting close to a 5″ phone.

Benefits of Smart Phones

Using a smart phone is good for teaching children dexterity. It can also be used for teaching art in situations where more traditional art forms such as finger painting aren’t possible (I have met a professional artist who has used a Samsung Galaxy Note phone for creating art work).

There is a huge range of educational apps for smart phones.

The Wikireader (that I reviewed 4 years ago) [4] has obvious educational benefits. But a phone with Internet access (either 3G or Wifi) gives Wikipedia access including all pictures and is a better fit for most pockets.

There are lots of educational web sites and random web sites that can be used for education (Googling the answer to random questions).

When it comes to preparing kids for “the real world” or “the work environment” people often claim that kids need to use Microsoft software because most companies do (regardless of the fact that most companies will be using radically different versions of MS software by the time current school kids graduate from university). In my typical work environment I’m expected to be able to find the answer to all sorts of random work-related questions at any time and I think that many careers have similar expectations. Being able to quickly look things up on a phone is a real work skill, and a skill that’s going to last a lot longer than knowing today’s version of MS-Office.

There are a variety of apps for tracking phones. There are non-creepy ways of using such apps for monitoring kids. Also with two-way monitoring kids will know when their parents are about to collect them from an event and can stay inside until their parents are in the area. This combined with the phone/SMS functionality that is available on feature-phones provides some benefits for child safety.

iOS vs Android

Rumour has it that iOS is better than Android for kids diagnosed with Low Functioning Autism. There are apparently apps that help non-verbal kids communicate with icons and for arranging schedules for kids who have difficulty with changes to plans. I don’t know anyone who has a LFA child so I haven’t had any reason to investigate such things. Anyone can visit an Apple store and a Samsung Experience store as they have phones and tablets you can use to test out the apps (at least the ones with free versions). As an aside the money the Australian government provides to assist Autistic children can be used to purchase a phone or tablet if a registered therapist signs a document declaring that it has a therapeutic benefit.

I think that Android devices are generally better for educational purposes than iOS devices because Android is a less restrictive platform. On an Android device you can install apps downloaded from a web site or from a 3rd party app download service. Even if you stick to the Google Play store there’s a wider range of apps to choose from because Google is apparently less restrictive.

Android devices usually allow installation of a replacement OS. The Nexus devices are always unlocked and have a wide range of alternate OS images and the other commonly used devices can usually have an alternate OS installed. This allows kids who have the interest and technical skill to extensively customise their device and learn all about it’s operation. iOS devices are designed to be sealed against the user. Admittedly there probably aren’t many kids with the skill and desire to replace the OS on their phone, but I think it’s good to have option.

Android phones have a range of sizes and features while Apple only makes a few devices at any time and there’s usually only a couple of different phones on sale. iPhones are also a lot smaller than most Android phones, according to my previous estimates of hand size the iPhone 5 would be a good tablet for a 3yo or good for side-grasp phone use for a 10yo [5]. The main benefits of a phone are for things other than making phone calls so generally the biggest phone that will fit in a pocket is the best choice. The tiny iPhones don’t seem very suitable.

Also buying one of each is a viable option.

Conclusion

I think that mobile phone ownership is good for almost all kids even from a very young age (there are many reports of kids learning to use phones and tablets before they learn to read). There are no real down-sides that I can find.

I think that Android devices are generally a better option than iOS devices. But in the case of special needs kids there may be advantages to iOS.

Related posts:

  1. Choosing an Android Phone My phone contract ends in a few months, so I’m...
  2. Standardising Android Don Marti wrote an amusing post about the lack of...
  3. My Ideal Mobile Phone Based on my experience testing the IBM Seer software on...

Sam Watkins: sswam

Mon, 2015-06-22 16:30

I learned a useful trick with the bash shell today.

We can use printf “%q ” to escape arguments to pass to the shell.

This can be useful in combination with ssh, in case you want to pass arguments containing shell special characters or spaces. It can also be used with su -c, and sh -c.

The following will run a command exactly on a remote server:

sshc() { remote=$1 ; shift         ssh "$remote" "`printf "%q " "$@"`" }

Example:

sshc user@server touch "a test file" "another file"

Sridhar Dhanapalan: Twitter posts: 2015-06-15 to 2015-06-21

Mon, 2015-06-22 00:27

Michael Still: Yet another possible cub walk

Sat, 2015-06-20 19:28
Jacqui and Catherine kindly agreed to come on another test walk for a possible cub walk. This one was the Sanctuary Loop at Tidbinbilla. To be honest this wasn't a great choice for cubs -- whilst being scenic and generally pleasant, the heavy use of black top paths and walkways made it feel like a walk in the Botanic Gardens, and the heavy fencing made it feel like an exhibit at a zoo. I'm sure its great for a weekend walk or for tourists, but if you're trying to have a cub adventure its not great.



                                       



See more thumbnails



Interactive map for this route.



Tags for this post: blog pictures 20150620-tidbinbilla photo canberra bushwalk

Related posts: Goodwin trig; Big Monks; Geocaching; Confessions of a middle aged orienteering marker; A quick walk through Curtin; Narrabundah trig and 16 geocaches



Comment