Planet Linux Australia

Syndicate content
Planet Linux Australia - http://planet.linux.org.au
Updated: 1 hour 48 min ago

OpenSTEM: This Week in HASS – term 2, week 9

Mon, 2017-06-12 09:03

The OpenSTEM™ Understanding Our World® units have only 9 weeks per term, so this is the last week! Our youngest students are looking at some Aboriginal Places; slightly older older students are thinking about what their school and local area were like when their parents and grandparents were children; and students in years 3 to 6 are completing their presentations and anything else that might be outstanding from the term.

Foundation/Prep/Kindy

Students in the stand-alone Foundation/Prep/Kindy class (Unit F.2) examine Aboriginal Places this week. Students examine which places are special to Aboriginal people, and how these places should be cared for by Aboriginal people and the broader community. Several of the Australian places in the Aunt Madge’s Suitcase Activity can be used to support this discussion in the classroom. Students in an integrated Foundation/Prep/Kindy and Year 1 class (Unit F.6), as well as Year 1 (Unit 1.2), 2 (Unit 2.2) and 3 (Unit 3.2) students consider life in the times of their parents and grandparents, with specific reference to their school, or the local area studied during this unit. Teachers may wish to invite older members of the community (including interested parents and/or grandparents) in to the class to describe their memories of the area in former years. Were any of them past students of the school? This is a great opportunity for students to come up with their own questions about life in past times.

Years 3 to 6 Aunt Madge

Students in Year 3 (Unit 3.6), 4 (Unit 4.2), 5 (Unit 5.2) and 6 (Unit 6.2) are finishing off their presentations and any outstanding work this week. Sometimes the middle of term can be very rushed and so it’s always good to have some breathing space at the end to catch up on anything that might have been squeezed out before. For those classes where everyone is up-to-date and looking for extra activities, the Aunt Madge’s Suitcase Activity is always popular with students and can be used to support their learning. Teachers may wish to select a range of destinations appropriate to the work covered during the term and encourage students to think about how those destinations relate to the material covered in class. Destinations may be selected by continent or theme – e.g. natural places or historical sites. A further advantage of Aunt Madge is that the activity can be tailored to fit the available time – from 5 or 10 minutes for a single destination, to 45 minutes or more for a full selection; and played in groups, or as a whole class, allowing some students to undertake the activity while other students may be catching up on other work. Students may also wish to revisit aspects of the Ancient Sailing Ships Activity and expand on their investigations.

Although this is the last week of this term’s units, we will have some more suggestions for extra activities next week – particularly those that keep the students busy while teachers attend to marking or compiling of reports.

Francois Marier: Mysterious 400 Bad Request in Django debug mode

Sun, 2017-06-11 10:21

While upgrading Libravatar to a more recent version of Django, I ran into a mysterious 400 error.

In debug mode, my site was working fine, but with DEBUG = False, I would only a page containing this error:

Bad Request (400)

with no extra details in the web server logs.

Turning on extra error logging

To see the full error message, I configured logging to a file by adding this to settings.py:

LOGGING = { 'version': 1, 'disable_existing_loggers': False, 'handlers': { 'file': { 'level': 'DEBUG', 'class': 'logging.FileHandler', 'filename': '/tmp/debug.log', }, }, 'loggers': { 'django': { 'handlers': ['file'], 'level': 'DEBUG', 'propagate': True, }, }, }

Then I got the following error message:

Invalid HTTP_HOST header: 'www.example.com'. You may need to add u'www.example.com' to ALLOWED_HOSTS. Temporary hack

Sure enough, putting this in settings.py would make it work outside of debug mode:

ALLOWED_HOSTS = ['*']

which means that there's a mismatch between the HTTP_HOST from Apache and the one that Django expects.

Root cause

The underlying problem was that the Libravatar config file was missing the square brackets around the ALLOWED_HOSTS setting.

I had this:

ALLOWED_HOSTS = 'www.example.com'

instead of:

ALLOWED_HOSTS = ['www.example.com']

Linux Users of Victoria (LUV) Announce: LUV Beginners June Meeting: Debian 9 release party!

Fri, 2017-06-09 19:02
Start: Jun 17 2017 12:30 End: Jun 17 2017 16:30 Start: Jun 17 2017 12:30 End: Jun 17 2017 16:30 Location:  Infoxchange, 33 Elizabeth St. Richmond Link:  http://luv.asn.au/meetings/map

Debian Linux version 9 (codename "Stretch") is scheduled for release on 17 June 2017.  Join us in celebrating the release and assisting anyone who would like to install or upgrade to the new version!

There will also be the usual casual hands-on workshop, Linux installation, configuration and assistance and advice. Bring your laptop if you need help with a particular issue. This will now occur BEFORE the talks from 12:30 to 14:00. The talks will commence at 14:00 (2pm) so there is time for people to have lunch nearby.

The meeting will be held at Infoxchange, 33 Elizabeth St. Richmond 3121 (enter via the garage on Jonas St.) Late arrivals, please call (0421) 775 358 for access to the venue.

LUV would like to acknowledge Infoxchange for the venue.

Linux Users of Victoria Inc., is an incorporated association, registration number A0040056C.

June 17, 2017 - 12:30

Lev Lafayette: Heredocs with Gaussian and Slurm

Thu, 2017-06-08 13:03

Gaussian is a well-known computational chemistry package, and sometimes subject to debate over its license (e.g., the terms state researchers who develop competing software packages are not permitted to use the software, compare performance etc). Whilst I have some strong opinions about such a license, this will be elaborated at another time. The purpose here is to illustrate the use of heredocs with Slurm.

read more

Danielle Madeley: Applied PKCS#11

Tue, 2017-06-06 23:01

The most involved thing I’ve had to learn this year is how to actually use PKCS #11 to talk to crypto hardware. It’s actually not that clear. Most of the examples are buried in random bits of C from vendors like Oracle or IBM; and the spec itself is pretty dense. Especially when it comes to understanding how you actually use it, and what all the bits and pieces do.

In honour of our Prime Minister saying he should have NOBUS access into our cryptography, which is why we should all start using hardware encryption modules (did you know you can use your TPM) and thus in order to save the next girl 6 months of poking around on a piece of hardware she doesn’t really *get*, I started a document: Applied PKCS#11.

The later sections refer to the API exposed by python-pkcs11, but the first part is generally relevant. Hopefully it makes sense, I’m super keen to get feedback if I’ve made any huge logical leaps etc.

Linux Users of Victoria (LUV) Announce: LUV Main June 2017 Meeting

Mon, 2017-06-05 13:02
Start: Jun 6 2017 18:30 End: Jun 6 2017 20:30 Start: Jun 6 2017 18:30 End: Jun 6 2017 20:30 Location:  The Dan O'Connell Hotel, 225 Canning Street, Carlton VIC 3053 Link:  http://luv.asn.au/meetings/map

PLEASE NOTE NEW LOCATION

Tuesday, June 6, 2017
6:30 PM to 8:30 PM
The Dan O'Connell Hotel
225 Canning Street, Carlton VIC 3053

Speakers:

• To be announced

Come have a drink with us and talk about Linux.  If you have something cool to show, please bring it along!

The Dan O'Connell Hotel, 225 Canning Street, Carlton VIC 3053

Food and drinks will be available on premises.

Before and/or after each meeting those who are interested are welcome to join other members for dinner.

Linux Users of Victoria Inc., is an incorporated association, registration number A0040056C.

June 6, 2017 - 18:30

OpenSTEM: This Week in HASS – term 2, week 8

Mon, 2017-06-05 09:04

This week we are starting into the last stretch of the term. Students are well into their final sections of work. Our youngest students are thinking about how we care for places, slightly older students are displaying their posters and older students are giving their presentations.

Foundation/Prep/Kindy to Year 3

Our youngest students doing the stand-alone Foundation/Prep/Kindy unit (F.2) are thinking about how we look after different places this week. Students in integrated Foundation/Prep/Kindy and Year 1 classes, doing Unit F.6, are displaying their posters on an issue in their local environment. These posters were prepared in proceeding weeks and can now be displayed either at school or in a local library or hall. The teacher may choose to invite parents to view the posters as well. Students in Years 1 (Unit 1.2), 2 (Unit 2.2) and 3 (Unit 3.2) also have posters to display on a range of issues, either at the school, in a local place, such as a park, or even a local heritage place. Discussions around points of view and the intended audience of the posters can help students to gain a more in-depth understanding and critique their own work.

Years 3 to 6

Students in Years 3 (Unit 3.6), 4 (Unit 4.2), 5 (Unit 5.2) and 6 (Unit 6.2) are in the second of 3 weeks set aside for their presentations. The presentations cover a significant body of work and thus a 3 weeks of lessons are set aside for the presentations, as well as for finishing any other sections of work not yet completed. Year 3 students are considering extreme climate areas of Australia and other parts of the world, such as the Sahara Desert, Arctic and Antarctica and Mount Everest, by studying explorers such as Edmund Hillary and Tenzing Norgay, Robert Scott and Pawel Strzelecki. Year 4 students are studying explorers and the environments and animals of Africa and South America, such as Francisco Pizarro, the Giant Vampire Bat, Vasco Da Gama and the Cape Lion. Year 5 students are studying explorers, environments and animals of North America, such as Henry Hudson, Hernando de Soto and the Great Auk. Year 6 students are studying explorers, environments and indigenous peoples of Asia, such as Vitus Bering, Zheng He, Marco Polo, the Mongols and the Rus.

Colin Charles: Speaking in June 2017

Sun, 2017-06-04 19:01

I will be at several events in June 2017:

  • db tech showcase 2017 – 16-17 June 2017 – Tokyo, Japan. I’m giving a talk about best practices around MySQL High Availability.
  • O’Reilly Velocity 2017 – 19-22 June 2017 – San Jose, California, USA. I’m giving a tutorial about best practices around MySQL High Availability. Use code CC20 for a 20% discount.

I look forward to meeting with you at either of these events, to discuss all things MySQL (High Availability, security, cloud, etc.), and how Percona can help you.

As I write this, I’m in Budva, Montenegro, for the Percona engineering meeting.

Ben Martin: Six is the magic number

Sun, 2017-06-04 17:01
I have talked about controlling robot arms with 4 or 5 motors and the maths involved in turning a desired x,y,z target into servo angles. Things get a little too interesting with 6 motors as you end up with a great deal of solutions to a positioning problem and need to work out a 'best' choice.


So I finally got MoveIt! to work to control a six motor arm using ROS. I now also know that using MoveIt on lower order arms isn't going to give you much love. Six is the magic number (plus claw motor) to get things working and patience is your best friend in getting the configuration and software setup going.

This was great as MoveIt was the last corner of the ROS stack that I hadn't managed to get to work for me. The great part is that the knowledge I gained playing with MoveIt will work on larger more accurate and expensive robot arms.

Lev Lafayette: The Why and How of HPC-Cloud Hybrids with OpenStack

Sat, 2017-06-03 17:03

High performance computing and cloud computing have traditionally been seen as separate solutions to separate problems, dealing with issues of performance and flexibility respectively. In a diverse research environment however, both sets of compute requirements can occur. In addition to the administrative benefits in combining both requirements into a single unified system, opportunities are provided for incremental expansion.

read more

Danielle Madeley: Update on python-pkcs11

Thu, 2017-06-01 23:01

I spent a bit of time fleshing out the support matrix for python-pkcs11 and getting things that aren’t SoftHSM into CI for integration testing (there’s still no one-command rollout for BuildBot connected to GitHub, but I got there in the end).

The nice folks at Nitrokey are also sending me some devices to widen the compatibility matrix. Also happy to make it work with CloudHSM if someone at Amazon wants to hook me up!

I also put together API docs that hopefully help to explain how to actually use the thing and added support for RFC3279 to pyasn1_modules (so you can encode your elliptic curve parameters).

Next goal is to open up my Django HSM integrations to add encrypted database fields, encrypted file storage and various other offloads onto the HSM. Also look at supporting certificate objects for all that wonderful stuff.

Stewart Smith: Fedora 25 + Lenovo X1 Carbon 4th Gen + OneLink+ Dock

Mon, 2017-05-29 13:00

As of May 29th 2017, if you want to do something crazy like use *both* ports of the OneLink+ dock to use monitors that aren’t 640×480 (but aren’t 4k), you’re going to need a 4.11 kernel, as everything else (for example 4.10.17, which is the latest in Fedora 25 at time of writing) will end you in a world of horrible, horrible pain.

To install, run this:

sudo dnf install \ https://kojipkgs.fedoraproject.org//packages/kernel/4.11.3/200.fc25/x86_64/kernel-4.11.3-200.fc25.x86_64.rpm \ https://kojipkgs.fedoraproject.org//packages/kernel/4.11.3/200.fc25/x86_64/kernel-core-4.11.3-200.fc25.x86_64.rpm \ https://kojipkgs.fedoraproject.org//packages/kernel/4.11.3/200.fc25/x86_64/kernel-cross-headers-4.11.3-200.fc25.x86_64.rpm \ https://kojipkgs.fedoraproject.org//packages/kernel/4.11.3/200.fc25/x86_64/kernel-devel-4.11.3-200.fc25.x86_64.rpm \ https://kojipkgs.fedoraproject.org//packages/kernel/4.11.3/200.fc25/x86_64/kernel-modules-4.11.3-200.fc25.x86_64.rpm \ https://kojipkgs.fedoraproject.org//packages/kernel/4.11.3/200.fc25/x86_64/kernel-tools-4.11.3-200.fc25.x86_64.rpm \ https://kojipkgs.fedoraproject.org//packages/kernel/4.11.3/200.fc25/x86_64/kernel-tools-libs-4.11.3-200.fc25.x86_64.rpm \ https://kojipkgs.fedoraproject.org//packages/kernel/4.11.3/200.fc25/x86_64/perf-4.11.3-200.fc25.x86_64.rpm

This grabs a kernel that’s sitting in testing and isn’t yet in the main repositories. However, I can now see things on monitors, rather than 0 to 1 monitor (most often 0). You can also dock/undock and everything doesn’t crash in a pile of fail.

I remember a time when you could fairly reliably buy Intel hardware and have it “just work” with the latest distros. It’s unfortunate that this is no longer the case, and it’s more of a case of “wait six months and you’ll still have problems”.

Urgh.

(at least Wayland and X were bug for bug compatible?)

Michael Still: Configuring docker to use rexray and Ceph for persistent storage

Mon, 2017-05-29 13:00
For various reasons I wanted to play with docker containers backed by persistent Ceph storage. rexray seemed like the way to do that, so here are my notes on getting that working...

First off, I needed to install rexray:

    root@labosa:~/rexray# curl -sSL https://dl.bintray.com/emccode/rexray/install | sh Selecting previously unselected package rexray. (Reading database ... 177547 files and directories currently installed.) Preparing to unpack rexray_0.9.0-1_amd64.deb ... Unpacking rexray (0.9.0-1) ... Setting up rexray (0.9.0-1) ... rexray has been installed to /usr/bin/rexray REX-Ray ------- Binary: /usr/bin/rexray Flavor: client+agent+controller SemVer: 0.9.0 OsArch: Linux-x86_64 Branch: v0.9.0 Commit: 2a7458dd90a79c673463e14094377baf9fc8695e Formed: Thu, 04 May 2017 07:38:11 AEST libStorage ---------- SemVer: 0.6.0 OsArch: Linux-x86_64 Branch: v0.9.0 Commit: fa055d6da595602715bdfd5541b4aa6d4dcbcbd9 Formed: Thu, 04 May 2017 07:36:11 AEST


Which is of course horrid. What that script seems to have done is install a deb'd version of rexray based on an alien'd package:

    root@labosa:~/rexray# dpkg -s rexray Package: rexray Status: install ok installed Priority: extra Section: alien Installed-Size: 36140 Maintainer: Travis CI User <travis@testing-gce-7fbf00fc-f7cd-4e37-a584-810c64fdeeb1> Architecture: amd64 Version: 0.9.0-1 Depends: libc6 (>= 2.3.2) Description: Tool for managing remote & local storage. A guest based storage introspection tool that allows local visibility and management from cloud and storage platforms. . (Converted from a rpm package by alien version 8.86.)


If I was building anything more than a test environment I think I'd want to do a better job of installing rexray than this, so you've been warned.

Next to configure rexray to use Ceph. The configuration details are cunningly hidden in the libstorage docs, and aren't mentioned at all in the rexray docs, so you probably want to take a look at the libstorage docs on ceph. First off, we need to install the ceph tools, and copy the ceph authentication information from the the ceph we installed using openstack-ansible earlier.

    root@labosa:/etc# apt-get install ceph-common root@labosa:/etc# scp -rp 172.29.239.114:/etc/ceph . The authenticity of host '172.29.239.114 (172.29.239.114)' can't be established. ECDSA key fingerprint is SHA256:SA6U2fuXyVbsVJIoCEHL+qlQ3xEIda/MDOnHOZbgtnE. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added '172.29.239.114' (ECDSA) to the list of known hosts. rbdmap 100% 92 0.1KB/s 00:00 ceph.conf 100% 681 0.7KB/s 00:00 ceph.client.admin.keyring 100% 63 0.1KB/s 00:00 ceph.client.glance.keyring 100% 64 0.1KB/s 00:00 ceph.client.cinder.keyring 100% 64 0.1KB/s 00:00 ceph.client.cinder-backup.keyring 71 0.1KB/s 00:00 root@labosa:/etc# modprobe rbd


You also need to configure rexray. My first attempt looked like this:

    root@labosa:/var/log# cat /etc/rexray/config.yml libstorage: service: ceph


And the rexray output sure made it look like it worked...

    root@labosa:/etc# rexray service start ● rexray.service - rexray Loaded: loaded (/etc/systemd/system/rexray.service; enabled; vendor preset: enabled) Active: active (running) since Mon 2017-05-29 10:14:07 AEST; 33ms ago Main PID: 477423 (rexray) Tasks: 5 Memory: 1.5M CPU: 9ms CGroup: /system.slice/rexray.service └─477423 /usr/bin/rexray start -f May 29 10:14:07 labosa systemd[1]: Started rexray.


Which looked good, but /var/log/syslog said:

    May 29 10:14:08 labosa rexray[477423]: REX-Ray May 29 10:14:08 labosa rexray[477423]: ------- May 29 10:14:08 labosa rexray[477423]: Binary: /usr/bin/rexray May 29 10:14:08 labosa rexray[477423]: Flavor: client+agent+controller May 29 10:14:08 labosa rexray[477423]: SemVer: 0.9.0 May 29 10:14:08 labosa rexray[477423]: OsArch: Linux-x86_64 May 29 10:14:08 labosa rexray[477423]: Branch: v0.9.0 May 29 10:14:08 labosa rexray[477423]: Commit: 2a7458dd90a79c673463e14094377baf9fc8695e May 29 10:14:08 labosa rexray[477423]: Formed: Thu, 04 May 2017 07:38:11 AEST May 29 10:14:08 labosa rexray[477423]: libStorage May 29 10:14:08 labosa rexray[477423]: ---------- May 29 10:14:08 labosa rexray[477423]: SemVer: 0.6.0 May 29 10:14:08 labosa rexray[477423]: OsArch: Linux-x86_64 May 29 10:14:08 labosa rexray[477423]: Branch: v0.9.0 May 29 10:14:08 labosa rexray[477423]: Commit: fa055d6da595602715bdfd5541b4aa6d4dcbcbd9 May 29 10:14:08 labosa rexray[477423]: Formed: Thu, 04 May 2017 07:36:11 AEST May 29 10:14:08 labosa rexray[477423]: time="2017-05-29T10:14:08+10:00" level=error msg="error starting libStorage server" error.driver=ceph time=1496016848215 May 29 10:14:08 labosa rexray[477423]: time="2017-05-29T10:14:08+10:00" level=error msg="default module(s) failed to initialize" error.driver=ceph time=1496016848216 May 29 10:14:08 labosa rexray[477423]: time="2017-05-29T10:14:08+10:00" level=error msg="daemon failed to initialize" error.driver=ceph time=1496016848216 May 29 10:14:08 labosa rexray[477423]: time="2017-05-29T10:14:08+10:00" level=error msg="error starting rex-ray" error.driver=ceph time=1496016848216


That's because the service is called rbd it seems. So, the config file ended up looking like this:

    root@labosa:/var/log# cat /etc/rexray/config.yml libstorage: service: rbd rbd: defaultPool: rbd


Now to install docker:

    root@labosa:/var/log# sudo apt-get update root@labosa:/var/log# sudo apt-get install linux-image-extra-$(uname -r) \ linux-image-extra-virtual root@labosa:/var/log# sudo apt-get install apt-transport-https \ ca-certificates curl software-properties-common root@labosa:/var/log# curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add - root@labosa:/var/log# sudo add-apt-repository \ "deb [arch=amd64] https://download.docker.com/linux/ubuntu \ $(lsb_release -cs) \ stable" root@labosa:/var/log# sudo apt-get update root@labosa:/var/log# sudo apt-get install docker-ce


Now let's make a rexray volume.

    root@labosa:/var/log# rexray volume ls ID Name Status Size root@labosa:/var/log# docker volume create --driver=rexray --name=mysql \ --opt=size=1 A size of 1 here means 1gb mysql root@labosa:/var/log# rexray volume ls ID Name Status Size rbd.mysql mysql available 1


Let's start the container.

    root@labosa:/var/log# docker run --name some-mysql --volume-driver=rexray \ -v mysql:/var/lib/mysql -e MYSQL_ROOT_PASSWORD=my-secret-pw -d mysql Unable to find image 'mysql:latest' locally latest: Pulling from library/mysql 10a267c67f42: Pull complete c2dcc7bb2a88: Pull complete 17e7a0445698: Pull complete 9a61839a176f: Pull complete a1033d2f1825: Pull complete 0d6792140dcc: Pull complete cd3adf03d6e6: Pull complete d79d216fd92b: Pull complete b3c25bdeb4f4: Pull complete 02556e8f331f: Pull complete 4bed508a9e77: Pull complete Digest: sha256:2f4b1900c0ee53f344564db8d85733bd8d70b0a78cd00e6d92dc107224fc84a5 Status: Downloaded newer image for mysql:latest ccc251e6322dac504e978f4b95b3787517500de61eb251017cc0b7fd878c190b


And now to prove that persistence works and that there's nothing up my sleeve...
    root@labosa:/var/log# docker run -it --link some-mysql:mysql --rm mysql \ sh -c 'exec mysql -h"$MYSQL_PORT_3306_TCP_ADDR" \ -P"$MYSQL_PORT_3306_TCP_PORT" -uroot -p"$MYSQL_ENV_MYSQL_ROOT_PASSWORD"' mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 3 Server version: 5.7.18 MySQL Community Server (GPL) Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> show databases; +--------------------+ | Database | +--------------------+ | information_schema | | mysql | | performance_schema | | sys | +--------------------+ 4 rows in set (0.00 sec) mysql> create database demo; Query OK, 1 row affected (0.03 sec) mysql> use demo; Database changed mysql> create table foo(val char(5)); Query OK, 0 rows affected (0.14 sec) mysql> insert into foo(val) values ('a'), ('b'), ('c'); Query OK, 3 rows affected (0.08 sec) Records: 3 Duplicates: 0 Warnings: 0 mysql> select * from foo; +------+ | val | +------+ | a | | b | | c | +------+ 3 rows in set (0.00 sec)


Now let's re-create the container and prove the data remains.

    root@labosa:/var/log# docker stop some-mysql some-mysql root@labosa:/var/log# docker rm some-mysql some-mysql root@labosa:/var/log# docker run --name some-mysql --volume-driver=rexray \ -v mysql:/var/lib/mysql -e MYSQL_ROOT_PASSWORD=my-secret-pw -d mysql 99a7ccae1ad1865eb1bcc8c757251903dd2f1ac7d3ce4e365b5cdf94f539fe05 root@labosa:/var/log# docker run -it --link some-mysql:mysql --rm mysql \ sh -c 'exec mysql -h"$MYSQL_PORT_3306_TCP_ADDR" -\ P"$MYSQL_PORT_3306_TCP_PORT" -uroot -p"$MYSQL_ENV_MYSQL_ROOT_PASSWORD"' mysql: [Warning] Using a password on the command line interface can be insecure. Welcome to the MySQL monitor. Commands end with ; or \g. Your MySQL connection id is 3 Server version: 5.7.18 MySQL Community Server (GPL) Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved. Oracle is a registered trademark of Oracle Corporation and/or its affiliates. Other names may be trademarks of their respective owners. Type 'help;' or '\h' for help. Type '\c' to clear the current input statement. mysql> use demo; Reading table information for completion of table and column names You can turn off this feature to get a quicker startup with -A Database changed mysql> select * from foo; +------+ | val | +------+ | a | | b | | c | +------+ 3 rows in set (0.00 sec)
So there you go.

Tags for this post: docker ceph rbd rexray
Related posts: So you want to setup a Ceph dev environment using OSA; Juno nova mid-cycle meetup summary: containers

Comment

David Rowe: LilacSat-1 Codec 2 in Space!

Mon, 2017-05-29 11:03

On May 25th LilacSat-1 was launched from the ISS. The exiting news is that it contains an analog FM to Codec 2 repeater. I’ve been in touch with Wei Mingchuan, BG2BHC during the development phase, and it’s wonderful to see the satellite in orbit. He reports that some Hams have had preliminary contacts.

The LilacSat-1 team have developed their own waveform, that uses a convolutional code running over BPSK at 9600 bit/s. Wei reports a MDS of about -127 dBm on a USRP B210 SDR which is quite respectable and much better than analog FM. GNU radio modules are available to support reception. I think it’s great that Wei and team have used open source (including Codec 2) to develop their own novel systems, in this case a hybrid FM/digital system with custom FEC and modulation.

Now I need to get organised with some local hams and find out how to work this satellite myself!

Part 2 – Making a LilacSat-1 Contact

On Saturday 3 June 2017 Mark VK5QI, Andy VK5AKH and I just made our first LilacSat-1 contact at 12:36 local time on a lovely sunny winter day here in Adelaide! Mark did a fine job setting up a receive station in his car, and Andy put together the video below showing both ends of the conversation:

The VHF tx and UHF rx stations were only 20m apart but the path to LilacSat-1 was about 400km each way. Plenty of signal as you can see from the error free scatter diagram.

I’m fairly sure there is something wrong with the audio (perhaps levels into the codec), as the decoded Codec 2 1300 bit/s signal is quite distorted. I can also hear similar distortion on other LilicSat-1 contacts I have listened too.

Let me show you what I mean. Here is a sample of my voice from LilacSat-1, and another sample of my voice that I encoded locally using the Codec 2 c2enc/c2dec command line tools.

There is a clue in this QSO – one end of the contact is much clearer than the other:

I’ll take a closer look at the Codec 2 bit stream from the satellite over the next few days to see if I can spot any issues.

Well done to LilacSat-1 team – quite a thrill for me to send my own voice through my own codec into space and back!

Part 3 – Level Analysis

Sunday morning 4 June after a cup of coffee! I added a little bit of code to codec2.c:codec2_decode_1300() to dump the energy quantister levels:

e_index = unpack_natural_or_gray(bits, &nbit, E_BITS, c2->gray); e[3] = decode_energy(e_index, E_BITS); fprintf(stderr, "%d %f\n", e_index, e[3]);

The energy of the current frame is encoded as a 5 bit binary number. It’s effectively the “AF gain” or “volume” of the current 40ms frame of speech. We unpack the bits and use a look up table to get the actual energy.

We can then run the Codec 2 command line decoder with the LilacSat-1 Codec 2 data Mark captured yesterday to extract a file of energies:

./c2dec 1300 ~/Desktop/LilacSat-1/lilacsat_dgr.c2 - 2>lilacsat1_energy.txt | play -t raw -r 8000 -s -2 - trim 30 6

The lilacsat1_energy.txt file contains the energy quantiser index and decoded energy in a table (matrix) that I can load into Octave and plot. I also ran the same text on the reference cq_freedv file used in Part 2 above:

So the top plot is the input speech “cq freedv ….”, and the middle plot the resulting energy quantiser index values. The energy bounces about with the level of the input speech. Now the bottom plot is from the LilacSat-1 sample. It is “red lined” – hard up against the upper limits of the quantiser. This could explain the audio distortion we are hearing.

Wei emailed me overnight and other Hams (e.g. Bob N6RFM) have discovered that reducing the Mic gain on the uplink FM radios indeed improves the audio quality. Wei is looking into in-flight adjustments of the gain between the FM rx and Codec 2 tx on LilacSat-1.

Note to self – I should look into quantiser ranges to make Codec 2 robust to people driving it with different levels.

Part 4 – Some Improvements

Sunday morning 4 June 11:36am pass: Mark set up his VHF tx in my car, and we played the cq_freedv canned wave file using a laptop and signalink so we could easily vary the tx drive:

Fortunately I have plenty of power available in my Electric Vehicle – we just tapped across 13.2V worth of Lithium cells in the rear pack:

We achieved better results, but not quite as good as using the source file directly without a journey through the VHF FM uplink:

LilacSat-1 3 June high mic gain

LilacSat-1 4 June low mic gain

encoded locally (no VHF FM uplink)

There is still quite a lot of noise on the decoded audio, probably from the VHF uplink. Codec 2 performs poorly in the presence of high levels of background noise. As we are under-deviating, the SNR of the FM uplink will be reduced, further increasing noise. However Wei has just emailed me that his team is reducing the “AF gain” between the VHF rx and Codec 2 on LilacSat-1 so we should hear some improvements on the next few passes.

Note to self #2 – add some noise reduction inside of Codec 2 to make it more robust to different input signal conditions.

Links

The LilacSat-1 page has links to GNU Radio modules that can be used to receive signals from the satellite.

Mark, VK5QI, describes he car’s exotic antennas system and how it was used on todays LilacSat-1 contact.

LilacSat-1 HowTo, Mark and I have documented the set up procedure for LilacSat-1, and written some scripts to help automate the process.

OpenSTEM: This Week in HASS – term 2, week 7

Mon, 2017-05-29 09:03

This week students are starting to round off their main body of assessable work for the term. Older students are completing and starting to present their presentations, while younger students have posters and models to finish off.

Foundation/Prep/Kindy to Year 3

Students in our stand-alone Foundation/Prep/Kindy unit (F.2) are continuing to explore with their senses this week. While still working on their model or collage for their Favourite Place, they are using their sense of Smell to consider which aromas they like or dislike. Teachers (and students) can bring in a range of things with different smells to explore in class. Ideas for these are given in the Teacher’s Handbook. An important part of this investigation is considering how one can represent one’s favourite smells in the model or collage – students might try to draw the objects associated with the smells, or see if they can find creative alternatives to represent this sense.

Students in integrated Foundation/Prep/Kindy and Year 1 classes (Unit F.6) and those in Years 1 (Unit 1.2), 2 (Unit 2.2) and 3 (Unit 3.2) are completing their posters to be displayed next week. These posters cover topics of local significance – either local history information, or dealing with issues, such as littering or the need for play equipment. As the students work on the posters, teachers are holding discussions with them on responsibility for different issues. The delegation of responsibility to members of the community, local government, other authorities, people who use the facilities, the school P&C, the students etc. should be discussed in class, so that students start to understand how people have different responsibilities in different situations. The teacher can also revisit issues of responsibility in the classroom – what are the students responsible for? What is the teacher responsible for? What is the school responsible for? These discussions are an important means of allowing the students to interact and practise group discussion skills, as well as helping them to think about responsibilities.

Years 3 to 6

Students in Year 3 (Unit 3.6) are completing their presentation on an extreme climate explorer and may start presenting it to the class this week. Year 4 students (Unit 4.2) are presenting on their explorer of Africa and South America. Year 5 students (Unit 5.2) are presenting on their chosen explorer from North America; and Year 6 students (Unit 6.2) are presenting on their chosen explorer from Asia. The remaining 3 weeks of this unit are allocated to the presentations, to ensure enough time for these to be given in full. The presentations should cover all the aspects raised over previous weeks and answered in the Student Workbook – the environments and geography of the areas explored; sustainability issues, such as extinction of animals and changes in local environments; characteristics of the countries involved in the explorations; reasons for explorations and how these created the background which led to the settlement of Australia and the role of indigenous people, as well as impact on indigenous people and their environments. The presentation is thus a comprehensive body of work.

 

David Rowe: FreeDV 2017 Road Map

Mon, 2017-05-29 09:03

Half way through the year but I thought I better write down some plans anyway! Helps me organise my thoughts and minimise the tangential work. The main goal for 2017 is a FreeDV mode that is competitive with SSB at low SNRs on HF channels. But first, lets see what happened in 2016….

Achievements in 2016

Here is the 2016 Roadmap. Reviewing it, we actually made good progress on a bunch of the planned activities:

  • Brady O’Brien (KC9TPA) worked with me to develop the FreeDV 2400A and 2400B modes.
  • Fine progress on the SM2000 project, summarised nicely in my Gippstech 2016 SM2000 talk. Thanks in particular to Brady, Rick (KA8BMA) and Neil (VK5KA).
  • The open telemetry work provided some key components for the Wenet system for high speed SSDV images from High Altitude Balloons. This work spun out of FSK modem development by Brady and myself, combined with powerful LDPC FEC codes from VK5DSP, with lots of work by Mark VK5QI, and AREG club members. It operates close to the limits of physics: with a 100mW signals we transmit HD images over 100km at 100 kbit/s using a $20 SDR and a good LNA. Many AREG members have set up Wenet receive stations using $100 roadkill laptops refurbished with Linux. It leaves commercial telemetry chips-sets in the dust, about 10dB behind us in terms of performance.
  • Ongoing FreeDV outreach via AREG FreeDV broadcasts, attending conferences and Hamfests. Thank you to all those who promote and use FreeDV.

FreeDV 2017 Roadmap

Codec 2 700C is the breakthrough I have been waiting for. Communications quality, conversational speech at just 700 bit/s, and even on a rough first pass it outperforms MELPe at 600 bit/s. Having a viable codec at 700 bit/s lets us consider powerful LDPC FEC codes in the 2000Hz SSB type bandwidths I’m targeting, which has led to a new OFDM modem and the emerging FreeDV 700D mode. I now feel comfortable that I can reach the goal of sub zero dB SNR digital speech that exceeds SSB in quality.

So here is the 2017 roadmap. Partially shaded work packages are partially complete. The pink work packages are ongoing activities rather than project based:

Rather than push FreeDV 700D straight out, I have decided to have another iteration at Codec 2 quality, using the Codec 700C algorithms as a starting point. The FreeDV 700D work has suggested we can use latency to overcome the HF channel, which means frames of several hundred ms to several seconds. By exploring correlation over longer Codec 2 frames we can achieve lower bit rates (e.g. sub 400 bit/s) or get better voice quality at 700 bit/s and above.

I’ve been knocking myself out to get good results at low SNRs. However many HF and indeed VHF/UHF PTT radio conversions take place at SNRs of greater than 10dB. This allows us to support higher bit rate codecs, and achieve better speech quality. For example moving from 0dB to 10dB means 10 times the bit rate at the same Bit Error Rate (BER). The OFDM modem will allow us to pack up to 4000 bit/s into a 2000 Hz SSB channel.

The algorithms that work so well for Codec 2 700C can be used to increase the quality at higher bit rates. So the goal of the “Codec 2 Quality” work package is to (i) improve the quality at 700-ish bit/s, and (ii) come up with a Codec 2 mode that improves on the speech quality of Codec 2 1300 (as used in the FreeDV 1600 mode) at 2000-ish bit/s.

After the Codec 2 quality improvement I will port the new algorithms to C, release and tune on the FreeDV GUI program, then port to the SM1000. Fortunately the new OFDM modem is simpler in terms of memory and computation that the COHPSK modem used for FreeDV 700C. We have an option to use a short LDPC code (224 bits) which with a little work will run OK on the SM1000.

Putting it all Together

The outputs will be a low SNR mode competitive with SSB at low SNR, and (hopefully) a high SNR mode that sounds better than SSB at medium to high SNRs. It will be available as a free software download (FreeDV GUI program), an embedded stand-alone product (the SM1000), and as a gcc library (FreeDV API)

Then we can get back to VHF/UHF work and the SM2000 project.

Help Wanted

When will this all happen? Much sooner if you help!

I’m a busy guy, making steady progress in the field of open source digital radio. While I appreciate your ideas, and enjoy brainstorming as much as the next person, what I really want are your patches, and consistent week in/week out effort. If you can code in C and/or have/are willing to learn a little GNU Octave, there is plenty of work to be done in SM1000 maintenance, a port of the OFDM modem to C, SM1000 hardware/software maintenance, and FreeDV GUI program refactoring and maintenance. Email me.

Links

FreeDV 2016 Roadmap. Promises, promises……
FreeDV 2400A and 2400B modes
SM2000 – First post introducing the SM2000 project
Gippstech 2016 SM2000 talk – Good summary of SM2000 project to date
LowSNR site – from Bill (VK5DSP) modem guru and LDPC code-smith
Horus 39 – Fantastic High Speed SSDV Images – Good summary of Wenet blog posts and modem technology
SM1000 Digital Voice Adaptor
AREG FreeDV broadcasts

Michael Still: So you want to setup a Ceph dev environment using OSA

Sun, 2017-05-28 13:00
Support for installing and configuring Ceph was added to openstack-ansible in Ocata, so now that I have a need for a Ceph development environment it seems logical that I would build it by building an openstack-ansible Ocata AIO. There were a few gotchas there, so I want to explain the process I used.

First off, Ceph is enabled in an openstack-ansible AIO using a thing I've never seen before called a "Scenario". Basically this means that you need to export an environment variable called "SCENARIO" before running the AIO install. Something like this will do the trick?L:

    export SCENARIO=ceph


Next you need to set the global pg_num in the ceph role or the install will fail. I did that with this patch:

    --- /etc/ansible/roles/ceph.ceph-common/defaults/main.yml 2017-05-26 08:55:07.803635173 +1000 +++ /etc/ansible/roles/ceph.ceph-common/defaults/main.yml 2017-05-26 08:58:30.417019878 +1000 @@ -338,7 +338,9 @@ # foo: 1234 # bar: 5678 # -ceph_conf_overrides: {} +ceph_conf_overrides: + global: + osd_pool_default_pg_num: 8 ############# @@ -373,4 +375,4 @@ # Set this to true to enable File access via NFS. Requires an MDS role. nfs_file_gw: true # Set this to true to enable Object access via NFS. Requires an RGW role. -nfs_obj_gw: false \ No newline at end of file +nfs_obj_gw: false


That of course needs to be done after the Ceph role has been fetched, but before it is executed, so in other words after the AIO bootstrap, but before the install.

And that was about it (although of course that took a fair while to work out). I have this automated in my little install helper thing, so I'll never need to think about it again which is nice.

Once Ceph is installed, you interact with it via the monitor container, not the utility container, which is a bit odd. That said, all you really need is the Ceph config file and the Ceph utilities, so you could move those elsewhere.

    root@labosa:/etc/openstack_deploy# lxc-attach -n aio1_ceph-mon_container-a3d8b8b1 root@aio1-ceph-mon-container-a3d8b8b1:/# ceph -s cluster 24424319-b5e9-49d2-a57a-6087ab7f45bd health HEALTH_OK monmap e1: 1 mons at {aio1-ceph-mon-container-a3d8b8b1=172.29.239.114:6789/0} election epoch 3, quorum 0 aio1-ceph-mon-container-a3d8b8b1 osdmap e20: 3 osds: 3 up, 3 in flags sortbitwise,require_jewel_osds pgmap v36: 40 pgs, 5 pools, 0 bytes data, 0 objects 102156 kB used, 3070 GB / 3070 GB avail 40 active+clean root@aio1-ceph-mon-container-a3d8b8b1:/# ceph osd tree ID WEIGHT TYPE NAME UP/DOWN REWEIGHT PRIMARY-AFFINITY -1 2.99817 root default -2 2.99817 host labosa 0 0.99939 osd.0 up 1.00000 1.00000 1 0.99939 osd.1 up 1.00000 1.00000 2 0.99939 osd.2 up 1.00000 1.00000


Tags for this post: openstack osa ceph openstack-ansible
Related posts: Configuring docker to use rexray and Ceph for persistent storage

Comment

OpenSTEM: This Week in HASS, term 2 week 5

Thu, 2017-05-25 17:04

NAPLAN’s over and it’s time to sink our teeth into the main body of curriculum work before mid-year reporting rolls around. Our younger students are using all their senses to study the environment and local area around them, whilst our older students are hard at work on their Explorer projects.

Foundation/Prep/Kindy to Year 3

Unit F.2 for stand-alone Foundation/Prep/Kindy classes has the students continuing to think about their Favourite Place. This week students are considering what they can hear in their Favourite Place and how they will depict that in their model of their Favourite Place. Students can also think about what their Favourite Sounds are and whether or not these would occur in their Favourite Place. Students in integrated Foundation/Prep/Kindy classes (Unit F.6) and Years 1 (Unit 1.2), 2 (Unit 2.2) and 3 (Unit 3.2) have this week set aside for an excursion to a local park or area of heritage significance. If an excursion outside school grounds is impractical teachers can achieve similar results from an excursion around the school and oval. Students are using their senses to interpret their environment, as well as thinking about living and non-living things, natural and managed landscapes and sources of heat and light.

Years 3 to 6

Students in Years 3 (Unit 3.6), 4 (Unit 4.2), 5 (Unit 5.2) and 6 (Unit 6.2) are continuing their project on an explorer. This week the focus for most students is on animals which may have been encountered by their explorer. Year 3 students are examining animals from different climate zones and how they are adapted to deal with climate extremes. Students in Years 4 and 5 look at extinct animals from Africa, South America or North America, assessing impact and sustainability issues. Students in Year 4 (and optionally as an extension for Year 3) consider the life cycle of their chosen animal. Students in years 4, 5 and 6 also start to examine the differences between Primary and Secondary sources and some of the OpenSTEM resources contain quotes or copies of primary material, so that students can refer to these in their project. Year 6 students are examining the changing Economies and Politics of Asia through time, in order to place the explorations within a broader context and to gain a greater understanding of the development of the global situation. Students have another 2 weeks to complete their presentation on their explorer (including environment and other aspects), before assessment of this project.

David Rowe: Urban HF Noise

Thu, 2017-05-25 13:03

Over the past 30 years, HF radio noise in urban areas has steadily increased. S6-S9 noise levels are common, which makes it hard to listen to the signals we want to receive.

I’ve been wondering if we can attenuate this noise using knowledge of the properties of the noise, and some clever DSP. Even 6dB would be useful, that’s like the transmitting station increasing their power by a factor of 4. I’ve just spent 2 months working on a 4dB improvement in my FreeDV work. So this week I’ve been messing about with pen and paper and a few simulations, exploring the problem of man-made noise on HF radio.

PWM Noise

One source of noise is switching power supplies, which have short, high current pulses flowing through them at a rate of a few hundred kHz. A series of short impulses in the time domain produces a series of spectral lines (i.e sinusoids or tones) in the frequency domain, so a 200kHz switcher produces tones at 200kHz, 400kHz, 600kHz etc. These tones are the “birdies” we hear as we tune our HF radios. The shorter the pulses are, the higher in frequency they will extend.

Short pulses lead to efficient switch mode power supplies, which is useful for energy efficiency, and especially desirable for high power devices like electric car chargers and solar panel inverters. So the trend is shorter switching times, higher currents and therefore more HF noise.

The power supplies adjust the PWM pulse-width back and forth as they adjust to varying conditions, which introduces a noise component. This is similar to phase noise in oscillators, and causes a continuous noise floor to appear in addition to the tones. The birdies we can tune around, but the noise floor sets a limit on urban HF operations.

The Octave script impulse_noise.m was used to generate the plots in this post. Here is a plot of some PWM impulse samples (top), and the HF spectrum.

I’ve injected a “wanted” signal at 1MHz for comparison. Given a switcher frequency of 255kHz, with 0.1V impulse amplitude, the noise floor is -90dBV down, or about 10uV. This is S5-S6 level noise, assuming 0.1V impulse amplitude induced onto our antenna by local switcher noise, e.g. nearby house wiring, or the neighbors TV. These numbers seem reasonable and match what we hear in our receivers.

Single Pulses

Single, isolated pulses are an easier problem. Examples are lightning or man-made sources that produce pulses at a rate slower than the bandwidth of the signal we are interested in.

A single impulse produces a flat spectrum, so the noise at frequency f Hz is almost the same as the noise at frequency f+delta Hz, where delta is small. This means you can use the noise at frequencies next to the one you are interested in to estimate and remove the noise in your frequency of interest.

Here is an impulse that lasts two samples, the magnitude spectrum changes slowly, although the phase changes quickly due to the time offset of the impulse.

Turns out that if the impulse position is known, and most of the energy is confined to that impulse, we can make a reasonable estimate of the noise at one frequency, from the noise at adjacent frequencies. Below we estimate the phase and magnitude (green cross) of frequency bin H(k+1) (nearby blue cross) from bin H(k). I’ve actually plotted H(k-1), H(k), and H(k+1) for comparison. The error in the estimation is -44dB down, so that’s a lot of noise removed.

Unfortunately this gets harder when there are multiple impulses in the same time window, and I can’t work out how to remove noise is this case. However this idea might be useful for some classes of impulse noise.

Noise Blanker

Another idea I tried was “blanking” out the impulses, buy opening and closing a switch so that the impulses are not allowed into the receiver. This works OK when we have a wideband signal, but falls over when just a bandpass version is available. In the bandpass version the “pulse” is smeared over time and we are no longer able to gate it out.

There will also be problems dealing with multiple PWM signals, that have different timing and frequency.

I haven’t looked at samples of the RF received from any real world switcher signals yet. I anticipate the magnitude and phase of the switcher signal will be all over the place, due to some torturous transfer function between the switcher and the terminals of my receiver. Plus various other signals will be present. Possibly there is a wide spectrum (short noise pulses) that we can work with. However I’d much rather deal with narrow bandpass signals consisting of just our wanted signal plus the switcher noise floor.

Next Steps

I might get back to my FreeDV work now, and leave this work on the back burner. I do feel I’m getting my head around the problem, and developing a “bag of tricks” that will be useful when other pieces fall into place.

The urban noise appears to be localised, e.g. if you head out into the country the background noise level is much lower. This suggests it’s coupled into the HF antenna by some local effect like induction. So another approach is to estimate the noise using a separate receiver that just picks up the local noise, through a sense antenna that is inefficient for long distance HF signals.

The local noise sequence could then be subtracted from the HF signal. I am aware of analog boxes that do this, using a magnitude and phase network to match the differences in signals received by the sense and HF antennas.

However a DSP approach will allow a more complex relationship (like an impulse response that extends for several microseconds) between the two antenna signals, and allow automatic adjustment. The noise spectrum can change quickly, as PWM is modulated and multiple devices turn on and off in the neighborhood. However the relationship between the two antennas will change slowly if they are fixed in space. This problem reminds me of echo cancellation, something I have played with before. Given radio hardware is now very cheap ($20 SDR dongles), multiple receivers could also be used.

So my gut feel remains that HF urban noise can be reduced to some extent (e.g. 6 or 12dB suppression) using DSP. If those nasty PWM switchers are inducing RF voltages into our antennas, we can work out a way to subtract those voltages.

Danielle Madeley: Announcing new high-level PKCS#11 HSM support for Python

Mon, 2017-05-22 23:02

Recently I’ve been working on a project that makes use of Thales HSM devices to encrypt/decrypt data. There’s a number of ways to talk to the HSM, but the most straight-forward from Linux is via PKCS#11. There were a number of attempts to wrap the PKCS#11 spec for Python, based on SWIG, cffi, etc., but they were all (a) low level, (b) not very Pythonic, (c) have terrible error handling, (d) broken, (e) inefficient for large files and (f) very difficult to fix.

Anyway, given that nearly all documentation on how to actually use PKCS#11 has to be discerned from C examples and thus I’d developed a pretty good working knowledge of the C API, and I’ve wanted to learn Cython for a while, I decided I’d write a new binding based on a high level wrapper I’d put into my app. It’s designed to be accessible, pick sane defaults for you, use generators where appropriate to reduce work, stream large files, be introspectable in your programming environment and be easy to read and extend.

https://github.com/danni/python-pkcs11

It’s currently a work in progress, but it’s now available on pip. You can get a session on a device, create a symmetric key, find objects, encrypt and decrypt data. The Cryptoki spec is quite large, so I’m focusing on the support that I need first, but it should be pretty straightforward for anyone who wanted to add something else they needed. I like to think I write reasonably clear, self-documenting code.

At the moment it’s only tested on SoftHSMv2 and the Thales nCipher Edge, which is what I have access to. If someone at Amazon wanted this to work flawlessly on CloudHSM, send me an account and I’ll do it Then I can look at releasing my Django integrations for fields, storage, signing, etc.