Planet Linux Australia
Justin, VK7TW, has published a video of my SM2000 presentation at Gippstech, which was held in July 2016.
Brady O’Brien, KC9TPA, visited me in June. Together we brought the SM2000 up to the point where it is decoding FreeDV 2400A waveforms at 10.7MHz IF, which we demonstrate in this video. I’m currently busy with another project but will get back to the SM2000 (and other FreeDV projects) later this year.
Thanks Justin and Brady!
FreeDV and this video was also mentioned on this interesting Reddit post/debate from Gary KN4AQ on VHF/UHF Digital Voice – a peek into the future
Thanks to the joys of abandonware websites, you can play with some interesting things from the 1990s and before. One of those things is OS/2 Warp. Now, I had a go at OS/2 sometime in the 1990s after being warned by a friend that it was “pretty much impossible” to get networking going. My experience of OS/2 then was not revolutionary… It was, well, something else on a PC that wasn’t that exciting and didn’t really add a huge amount over Windows.
Now, I’m nowhere near insane enough to try this on my actual computer, and I’ve managed to not accumulate any ancient PCs….
Luckily, qemu helps with an emulator! If you don’t set your CPU to Pentium (or possibly something one or two generations newer) then things don’t go well. Neither does a disk that by today’s standards would be considered beyond tiny. Also, if you dare to try to use an unpartitioned hard disk – OH MY are you in trouble.
Why I can’t create a partition… WHO KNOWS. But, I tried again with a 750MB disk that already had a partition on it and…. FAIL. I think this one was due to partition type, so I tried again with partition type of 6 – plain FAT16, and not W95 FAT16 (LBA). Some memory is coming back to me of larger drives and LBA and nightmares…
But that worked!
It still asked me for some config, but I gleefully ignored it (because that must be safe, right!?) and then I needed to select a network adapter! Due to a poor choice on my part, I started with a rtl8139, which is conspicuously absent from this fine list of Token Ring adapters:
Today we’re remembering Seamour Papert, as we’ve received news that he died a few days ago (31st July 2016) at the age of 88. Throughout his life, Papert did so much for computing and education, he even worked with the famous Jean Piaget who helped Papert further develop his views on children and learning.
For us at OpenSTEM, Papert is also special because in the late 1960s (yep that far back) he invented the Logo programming language, used to control drawing “turtles”. The Mirobot drawing turtle we use in our Robotics Program is a modern descendant of those early (then costly) adventures.
I sadly never met him, but what a wonderful person he was.
For more information, see the media release at MIT’s Media Lab (which he co-founded) or search for his name online.
The somewhat nebulous term "supercomputer" has a long history. Although first coined in the 1920s to refer to IBMs tabulators, in electronic computing the most important initial contribution was the CDC6600 in the 1960s, due to its advanced performance over competitors. Over time major technological advancements included vector processing, cluster architecture, massive processors counts, GPGPU technologies, multidimensional torus architectures for interconnect.
I’ve been playing with Prometheus monitoring lately. It is fairly new software that is getting popular. Prometheus works using a pull architecture. A central server connects to each thing you want to monitor every few seconds and grabs stats from it.
In the simplest case you run the node_exporter on each machine which gathers about 600-800 (!) metrics such as load, disk space and interface stats. This exporter listens on port 9100 and effectively works as an http server that responds to “GET /metrics HTTP/1.1” and spits several hundred lines of:node_forks 7916 node_intr 3.8090539e+07 node_load1 0.47 node_load15 0.21 node_load5 0.31 node_memory_Active 6.23935488e+08
Other exporters listen on different ports and export stats for apache or mysql while more complicated ones will act as proxies for outgoing tests (via snmp, icmp, http). The full list of them is on the Prometheus website.
So my problem was that I wanted to check my virtual machine that is on Linode. The machine only has a public IP and I didn’t want to:
- Allow random people to check my servers stats
- Have to setup some sort of VPN.
So I decided that the best way was to just use put a user/password on the exporter.
However the node_exporter does not implement authentication itself since the authors wanted the avoid maintaining lots of security code. So I decided to put it behind a reverse proxy using apache mod_proxy.Step 1 – Install node_exporter
Node_exporter is a single binary that I started via an upstart script. As part of the upstart script I told it to listen on localhost port 19100 instead of port 9100 on all interfaces# cat /etc/init/prometheus_node_exporter.conf description "Prometheus Node Exporter" start on startup chdir /home/prometheus/ script /home/prometheus/node_exporter -web.listen-address 127.0.0.1:19100 end script
Once I start the exporter a simple “curl 127.0.0.1:19100/metrics” makes sure it is working and returning data.Step 2 – Add Apache proxy entry
First make sure apache is listening on port 9100 . On Ubuntu edit the /etc/apache2/ports.conf file and add the line:Listen 9100
Next create a simple apache proxy without authentication (don’t forget to enable mod_proxy too):# more /etc/apache2/sites-available/prometheus.conf <VirtualHost *:9100> ServerName prometheus CustomLog /var/log/apache2/prometheus_access.log combined ErrorLog /var/log/apache2/prometheus_error.log ProxyRequests Off <Proxy *> Allow from all </Proxy> ProxyErrorOverride On ProxyPass / http://127.0.0.1:19100/ ProxyPassReverse / http://127.0.0.1:19100/ </VirtualHost>
This simply takes requests on port 9100 and forwards them to localhost port 19100 . Now reload apache and test via curl to port 9100. You can also use netstat to see what is listening on which ports:Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 127.0.0.1:19100 0.0.0.0:* LISTEN 8416/node_exporter tcp6 0 0 :::9100 :::* LISTEN 8725/apache2
Step 3 – Get Prometheus working
I’ll assume at this point you have other servers working. What you need to do now is add the following entries for you server in you prometheus.yml file.
First add basic_auth into your scape config for “node” and then add your servers, eg:- job_name: 'node' scrape_interval: 15s basic_auth: username: prom password: mypassword static_configs: - targets: ['myserver.example.com:9100'] labels: group: 'servers' alias: 'myserver'
Now restart Prometheus and make sure it is working. You should see the following lines in your apache logs plus stats for the server should start appearing:10.212.62.207 - - [31/Jul/2016:11:31:38 +0000] "GET /metrics HTTP/1.1" 200 11377 "-" "Go-http-client/1.1" 10.212.62.207 - - [31/Jul/2016:11:31:53 +0000] "GET /metrics HTTP/1.1" 200 11398 "-" "Go-http-client/1.1" 10.212.62.207 - - [31/Jul/2016:11:32:08 +0000] "GET /metrics HTTP/1.1" 200 11377 "-" "Go-http-client/1.1"
Notice that connections are 15 seconds apart, get http code 200 and are 11k in size. The Prometheus server is using Authentication but apache doesn’t need it yet.Step 4 – Enable Authentication.
Now create an apache password file:htpasswd -cb /home/prometheus/passwd prom mypassword
and update your apache entry to the followign to enable authentication:# more /etc/apache2/sites-available/prometheus.conf <VirtualHost *:9100> ServerName prometheus CustomLog /var/log/apache2/prometheus_access.log combined ErrorLog /var/log/apache2/prometheus_error.log ProxyRequests Off <Proxy *> Order deny,allow Allow from all # AuthType Basic AuthName "Password Required" AuthBasicProvider file AuthUserFile "/home/prometheus/passwd" Require valid-user </Proxy> ProxyErrorOverride On ProxyPass / http://127.0.0.1:19100/ ProxyPassReverse / http://127.0.0.1:19100/ </VirtualHost>
After you reload apache you should see the following:10.212.56.135 - prom [01/Aug/2016:04:42:08 +0000] "GET /metrics HTTP/1.1" 200 11394 "-" "Go-http-client/1.1" 10.212.56.135 - prom [01/Aug/2016:04:42:23 +0000] "GET /metrics HTTP/1.1" 200 11392 "-" "Go-http-client/1.1" 10.212.56.135 - prom [01/Aug/2016:04:42:38 +0000] "GET /metrics HTTP/1.1" 200 11391 "-" "Go-http-client/1.1"
Note that the “prom” in field 3 indicates that we are logging in for each connection. If you try to connect to the port without authentication you will get:Unauthorized This server could not verify that you are authorized to access the document requested. Either you supplied the wrong credentials (e.g., bad password), or your browser doesn't understand how to supply the credentials required.
That is pretty much it. Note that will need to add additional Virtualhost entries for more ports if you run other exporters on the server.
Early days yet, but playing with NERSC’s Shifter to let us use Docker containers safely on our test RHEL6 cluster is looking really interesting (given you can’t use Docker itself under RHEL6, and if you could the security concerns would cancel it out anyway).
To use a pre-built Ubuntu Xenial image, for instance, you tell it to pull the image:[samuel@bruce ~]$ shifterimg pull ubuntu:16.04
There’s a number of steps it goes through, first retrieving the container from the Docker Hub:2016-08-01T18:19:57 Pulling Image: docker:ubuntu:16.04, status: PULLING
Then disarming the Docker container by removing any setuid/setgid bits, etc, and repacking as a Shifter image:2016-08-01T18:20:41 Pulling Image: docker:ubuntu:16.04, status: CONVERSION
…and then it’s ready to go:2016-08-01T18:21:04 Pulling Image: docker:ubuntu:16.04, status: READY
Using the image from the command line is pretty easy:[samuel@bruce ~]$ cat /etc/lsb-release LSB_VERSION=base-4.0-amd64:base-4.0-noarch:core-4.0-amd64:core-4.0-noarch:graphics-4.0-amd64:graphics-4.0-noarch:printing-4.0-amd64:printing-4.0-noarch [samuel@bruce ~]$ shifter --image=ubuntu:16.04 cat /etc/lsb-release DISTRIB_ID=Ubuntu DISTRIB_RELEASE=16.04 DISTRIB_CODENAME=xenial DISTRIB_DESCRIPTION="Ubuntu 16.04 LTS"
and the shifter runtime will copy in a site specified /etc/passwd, /etc/group and /etc/nsswitch.conf files so that you can do user/group lookups easily, as well as map in site specified filesystems, so your home directory is just where it would normally be on the cluster.[samuel@bruce ~]$ shifter --image=debian:wheezy bash --login samuel@bruce:~$ pwd /vlsci/VLSCI/samuel
I’ve not yet got to the point of configuring the Slurm plugin so you can queue up a Slurm job that will execute inside a Docker container, but very promising so far!
Correction: a misconception on my part – Shifter doesn’t put a Slurm batch job inside the container. It could, but there are good reasons why it’s better to leave that to the user (soon to be documented on the Shifter wiki page for Slurm integration).
This item originally posted here:
Playing with Shifter – NERSC’s tool to use Docker containers in HPC
This story is about the tectonic plate on which we reside. Tectonic plates move, and so continents shift over time. They generally go pretty slow though.
What about Australia? It appears that every year, we move 11 centimetres West and 7 centimetres North. For a tectonic plate, that’s very fast.
The last time scientists marked our location on the globe was in 1994, with the Geocentric Datum of Australia 1994 (GDA1994) – generally called GDA94 in geo-spatial tools (such as QGIS). So that datum came into force 22 years ago. Since then, we’ve moved an astonishing 1.5 metres! You may not think much of this, but right now it actually means that if you use a GPS in Australia to get coordinates, and plot it onto a map that doesn’t correct for this, you’re currently going to be off by 1.5 metres. Depending on what you’re measuring/marking, you’ll appreciate this can be very significant and cause problems.
Bear in mind that, within Australia, GDA94 is not wrong as such, as its coordinates are relative to points within Australia. However, the positioning of Australia in relation to the rest of the globe is now outdated. Positioning technologies have also improved. So there’s a new datum planned for Australia, GDA2020. By the time it comes into force, we’ll have shifted by 1.8 metres relative to GDA94.
We can have some fun with all this:
- If you stand and stretch both your arms out, the tips of your fingers are about 1.5 metres apart – of course this depends a bit on the length of your arms, but it’ll give you a rough idea. Now imagine a pipe or cable in the ground at a particular GPS position, move 1.5 metres. You could clean miss that pipe or cable… oops! Unless your GPS is configured to use a datum that gets updated, such as WGS84. However, if you had the pipe or cable plotted on a map that’s in GDA94, it becomes messy again.
- If you use a tool such as Google Earth, where is Australia actually? That is, will a point be plotted accurately, or be 1.5 metres out, or somewhere in between?
Well, that would depend on when the most recent broad scale photos were taken, and what corrections the Google Earth team possibly applies during processing of its data (for example, Google Earth uses a different datum – WGS 84 for its calculations).
Interesting question, isn’t it…
- Now for a little science/maths challenge. The Northern most tip of Australia, Cape York, is just 150km South of Papua New Guinea (PNG). Presuming our plate maintains its present course and speed, roughly how many years until the visible bits (above sea level) of Australia and PNG collide? Post your answer with working/reasoning in a comment to this post! Think about this carefully and do your research. Good luck!
Since my last post regarding the conversion of media from Channel 9’s Catch Up service, I have been in discussion with the company behind this technology, Hiro-Media. My concerns were primarily around their use of the open source xvid media codec and whilst I am not a contributor to xvid (and hence do not have any ownership under copyright), I believe it is still my right under the GPL to request a copy of the source code.
First off I want to thank Hiro-Media for their prompt and polite responses. It is clear that they take the issue of license violations very seriously. Granted, it would be somewhat hypocritical for a company specialising in DRM to not take copyright violations within their own company seriously, but it would not be the first time.
I initially asserted that, due to Hiro’s use (and presumed modification) of xvid code, that this software was considered a derivative and therefore bound in its entirety by the GPL. Hiro-Media denied this stating they use xvid in its original, unmodified state and hence Hiro is simply a user of rather than a derivative of xvid. This is a reasonable statement albeit one that is difficult to verify. I want to stress at this point that in my playing with the Hiro software I have NOT in anyway reverse engineered it nor have I attempted to decompile their binaries in any way.
In the end, the following points were revealed:
- The Mac version of Hiro uses a (claimed) unmodified version of the Perian Quicktime component
- The Windows version of Hiro currently on Channel 9’s website IS indeed modified, what Hiro-Media terms an ‘accidental internal QA’ version. They state that they have sent a new version to Channel 9 that corrects this. The xvid code they are using can be found at http://www.koepi.info/xvid.html
- Neither version has included a GPL preamble within their EULA as required. Again, I am assured this is to be corrected ASAP.
I want to reiterate that Hiro-Media have been very cooperative about this and appear to have genuine concern. I am impressed by the Hiro system itself and whilst I am still not a fan of DRM in general, this is by far the best compromise I have seen to date. They just didn’t have a linux version.
This brings me to my final, slightly more negative point. On my last correspondence with Hiro-Media, they concluded with the following:
Finally, please note our deepest concerns as to any attempt to misuse our system, including the content incorporated into it, as seems to be evidenced in your website. Prima facia, such behavior consists a gross and fundamental breach of our license (which you have already reviewed). Any such misuse may cause our company, as well as any of our partners, vast damages.
I do not wish to label this a threat (though I admit to feeling somewhat threatened by it), but I do want to clear up a few things about what I have done. The statement alleges I have violated Hiro’s license (pot? kettle? black?) however this is something I vehemently disagree with. I have read the license very careful (Obviously as I went looking for the GPL) and the only relevant part is:
You agree that you will not modify, adapt or translate, or disassemble, decompile, reverse engineer or otherwise attempt to discover the source code of the Software.
Now I admit to being completely guilty of a single part of this, I have attempted to discover the source code. BUT (and this is a really big BUT), I have attempted this by emailing Hiro-Media and asking them for it, NOT by decompiling (or in any other way inspecting) the software. In my opinion, the inclusion of that specific part in their license also goes against the GPL as such restrictions are strictly forbidden by it.
But back to the point, I have not modified, translated, disassembled, decompiled or reverse engineered the Hiro software. Additionally, I do not believe I have adapted it either. It is still doing exactly the same thing as it was originally, that is taking an incoming video stream, modifying it and decoding it. Importantly, I do not modify any files in any way. What I have altered is how Quicktime uses the data returned by Hiro. All my solution does is (using official OSX/Quicktime APIs) divert the output to a file rather than to the screen. In essence I have not done anything different to the ‘Save As’ option found in Quicktime Pro, however not owning Quicktime Pro, I merely found another way of doing this.
So that’s my conclusion. I will reply to Hiro-Media with a link to this post asking whether they still take issue with what I have done and take things from there.
To the guys from Hiro if you are reading this, I didn’t do any of this to start trouble. All I wanted was a way to play these files on my linux HTPC, with or without ads. Thankyou.
About 6 months ago, Channel 9 launched their ‘Catch Up’ service. Basically this is their way of fighting piracy and allowing people to download Australian made TV shows to watch on their PC. Now, of course, no ‘old media’ service would possibly do this without the wonders of DRM. Channel 9 though, are taking a slightly different approach.
Instead of the normal style of DRM that prevents you copying the file, Channel 9 employs technology from a company called Hiro. Essentially you install the Hiro player, download the file and watch it. The player will insert unskippable ads throughout the video, supposedly even targeted at your demographic. Now this is actually a fairly neat system, Channel 9 actually encourage you to share the video files over bittorrent etc! The problem, as I’m sure you can guess, is that there’s no player for linux.
So, just to skip to the punchline, yes it IS possible to get these files working on free software (completely legally & without the watermark)! If you just want to know how to do it, jump to the end as I’m going to explain a bit of background first.
The Hiro technology is interesting in that it isn’t simply some custom player. The files you download from Channel 9 are actually xvid encoded, albeit a bastard in-bred cousin of what xvid should be. If you simply download the file and play it with vlc or mplayer, it will run, however you will get a nasty watermark over the top of the video and it will almost certainly crash about 30s in when it hits the first advertising blob. There is also some trickiness going on with the audio as, even if you can get the video to keep playing, the audio will jump back to the beginning at this point. Of course, the watermark isn’t just something that’s placed over the top in post-processing like a subtitle, its in the video data itself. To remove it you actually need to filter the video to modify the area covered by the watermark to darken/lighten the pixels affected. Sounds crazy and a tremendous amount of work right? Well thankfully its already been done, by Hiro themselves.
When you install Hiro, you don’t actually install a media player, you install either a DirectShow filter or a Quicktime component depending on your platform. This has the advantage that you can use reasonably standard software to play the files. Its still not much help for linux though.
Before I get onto how to create a ‘normal’ xvid file, I just want to mention something I think should be a concern for free software advocates. As you might know, xvid is an open codec, both for encoding and decoding. Due to the limitations of Quicktime and Windows Media Player, Hiro needs to include an xvid decoder as part of their filter. I’m sure its no surprise to anyone though that they have failed to release any code for this filter, despite it being based off a GPL’d work. IA(definitely)NAL, but I suspect there’s probably some dodginess going on here.
Using Catchup with free software
Very basically, the trick to getting the video working is that it needs to be passed through the filter provided by Hiro. I tried a number of methods to get the files converted for use with mplayer or vlc and in the end, unfortunately, I found that I needed to be using either Windows or OSX to get it done. Smarter minds than mine might be able to get the DirectShow filter (HiroTransform.ax) working with mplayer in a similar manner to how CoreAVC on linux works, but I had no luck.
But, if you have access to OSX, here’s how to do it:
- Download and install the Hiro software for Mac. You don’t need to register or anything, in fact, you can delete the application the moment you finish the install. All you need is the Quicktime component it added.
- Grab any file from the Catch Up Service (http://video.ninemsn.com.au/catchuptv). I’ve tested this with Underbelly, but all videos should work.
- Install ffmpegx (http://ffmpegx.com)
- Grab the following little script: CleanCatch.sh
chmod +x CleanCatch.sh
- Voila. Output will be a file called ‘<filename>.clean.MP4′ and should be playable in both VLC and mplayer
So, I’m the first to admit that the above is a right royal pain to do, particularly the whole requiring OSX part. To save everyone the hassle though, I believe its possible to simply distribute the modified file. Now again, IANAL, but I’ve gone over the Channel 9 website with a fine tooth comb and can see nothing that forbids me from distributing this newly encoded file. I agreed to no EULA when I downloaded the original video and their site even has the following on it:
You can share the episode with your friends and watch it as many times as you like – online or offline – with no limitations
That whole ‘no limitations’ part is the bit I like. Not only have Channel 9 given me permission to distribute the file, they’ve given it to me unrestricted. I’ve not broken any locks and in fact have really only used the software provided by Channel 9 and a standard transcoding package.
This being the case, I am considering releasing modified versions of Channel 9’s files over bittorrent. I’d love to hear people’s opinions about this before doing so though in case they know more than I (not a hard thing) about such matters.
[Update: The code or the rallyduino can be found at: http://code.google.com/p/rallyduino/]
Amidst a million other things (Today is T-1 days until our little bubs technical due date! No news yet though) I finally managed to get the rallyduino into the car over the weekend. It actually went in a couple of weeks ago, but had to come out again after a few problems were found.
So, the good news, it all works! I wrote an automagic calibration routine that does all the clever number crunching and comes up with the calibration number, so after a quick drive down the road, everything was up and running. My back of the envelope calculations for what the calibration number for our car would be also turned out pretty accurate, only about 4mm per revolution out. The unit, once calibrated, is at least as accurate as the commercial devices and displays considerably more information. Its also more flexible as it can switch between modes dynamically and has full control from the remote handset. All in all I was pretty happy, even the instantaneous speed function worked first time.
To give a little background, the box uses a hall effect sensor mounted next to a brake caliper that pulses each time a wheel stud rotates past. This is a fairly common method for rally computers to use and to make life simpler, the rallyduino is pin compatible with another common commercial product, the VDO Minicockpit. As we already had a Minicockpit in the car, all we did was ‘double adapt’ the existing plug and pop in the rallyduino. This means there’s 2 computers running off the 1 sensor, in turn making it much simpler (assuming 1 of them is known to be accurate) to determine if the other is right.
The 4 devices in the pic are:
- Wayfinder electronic compass
- Terratrip (The black box)
- Rallyduino (Big silver box with the blue screen)
- VDO Minicockpit (Sitting on top of the rallyduino)
The major problem should be immediately obvious. The screen is completely unsuitable. Its both too small and has poor readability in daylight. I’m currently looking at alternatives and it seems like the simplest thing to do is get a 2×20 screen the same physical size as the existing 4×20. This, however, means that there would only be room for a single distance tracker rather than the 2 currently in place. The changeover is fairly trivial as the code, thankfully, is nice and flexible and the screen dimensions can be configured at compile time (From 2×16 to 4×20). Daylight readable screens are also fairly easily obtainable (http://www.crystalfontz.com is the ultimate small screen resource) so its just a matter of ordering one. In the long term I’d like to look at simply using a larger 4×20 screen but, as you can see, real estate on the dash is fairly tight.
Speaking of screens, I also found the most amazing little LCD controller from web4robot.com. It has both serial and I2C interfaces, a keypad decoder (with inbuilt debounce) and someone has gone to all the hard work of work of writing a common arduino interface library for it and other I2C LCD controllers (http://www.wentztech.com/radio/arduino/projects.html) . If you’re looking for such a LCD controller and you are working with an arduino, I cannot recommend these units enough. They’re available on eBay for about $20 Au delivered too. As much as I loved the unit from Phil Anderson, it simply doesn’t have the same featureset as this one, nor is it as robust.
So that’s where things are at. Apologies for the brain dump nature of this post, I just needed to get everything down while I could remember it all.
No updates to this blog in a while I’m afraid, things have just been far too busy to have had anything interesting (read geeky) to write about. That said, I am indulging and making this post purely to show off.
On Monday 25th May at 8:37am, after a rather long day/night, Mel and I welcomed Spencer Bailey Stewart into the world. There were a few little issues throughout the night (Particularly towards the end) and he had some small hurdles to get over in his first hour, but since then he has gone from strength to strength and both he and Mum and now doing amazingly well.
He’s a very placid little man and would quite happily sleep through an earthquake, much to our delight. And yes, that is a little penguin he’s holding on to in that pic
So that’s all really. I am very conscious of not becoming a complete baby bore so unless something actually worth writing about happens, this will hopefully be my only baby post for the sake of a baby post.
Just a quick post to say the ABC iView Boxee app has been updated to version 0.7 and should now be fully functional once again. I apologise to anyone who has been using the app for how long this update has taken and I wish I could say I’ve been off solving world hunger or something, but in reality I’ve just been flat out with work and family. I’ve also got a few other projects on the go that have been keeping me busy. These may or may not ever reach a stage of public consumption, but if they do it’ll be cool stuff.
For anyone using Boxee, you may need to remove the app from My Apps and wait for Boxee to refresh its repository index, but eventually version 0.7 should pop up. Its a few rough in places so I hope to do another cleanup within the next few weeks, but at least everything is functional again.
Lately my toying around with media centers has created some opportunities for commercial work in this area, which has been a pleasant change. As a result of this I’ve formed Noisy Media, a company specialising in the development of media centre apps for systems such as Boxee as well as the creation of customised media applications using XBMC (and similar) as a base.
Whilst I don’t expect this venture to ever be huge, I can see the market growing hugely in the future as products such as Google TV (Something I will be looking at VERY closely) and the Boxee Box are released and begin bringing streaming Video on Demand to the loungeroom. Given this is something I have a true passion for, the ability to turn work in this area into something profitable is very appealing, and exciting.
Here’s to video on demand!
Just a quick note to advise that the ASX RSS feed at http://noisymime.org/asx is currently not functional due to a change in the source data format. I am in the process of performing a rewrite on this now and should have it back up and running (Better and more robust than ever) within the next few days.
Apologies for the delay in getting things back up and running.
Besides being your run of the mill computer geek, I’ve always been a bit of a car geek as well. This often solicits down-the-nose looks from others who associate such people with V8 Supercar lovin’ petrolheads, which has always surprised me little because the most fun parts of working on a car are all just testing physics theories anyway. With that in mind, I’ll do this writeup from the point of view of the reader being a non-car, but scientifically minded person. First a bit of background…Background
For the last 3 years or so my dad and I have been working a project to fuel inject our race car. The car itself is a 1968 Mk2 Cortina and retains the original 40 year old 1600 OHV engine. This engine is originally carbureted, meaning that it has a device that uses the vacuum created by the engine to mix fuel and air. This mixture is crucial to the running of an engine as the ratio of fuel to air dramatically alters power, response and economy. Carburetors were used for this function for a long time and whilst they achieve the basics very well, they are, at best, a compromise for most engines. To overcome these limitations, car manufacturers started moving to fuel injection in the 80’s, which allowed precise control of the amount of fuel added through the use of electronic signals. Initially these systems were horrible however, being driven by analog or very basic digital computers that did not have the power or inputs needed to accurately perform this function. These evolved to something useful throughout the 90’s and by the 00’s cars were having full sequential system (more on this later) that could deliver both good performance and excellent economy. It was our plan to fit something like the late 90’s type systems (ohh how did this change by the end though) to the Cortina with the aims of improving the power and drivability of the old engine. IN this post I’m going to run through the various components needed from the electrical side to make this all happen, as well as a little background on each. Our starting point was this:
To have a computer control when are how much fuel to inject, it requires a number of inputs:
- A crank sensor. This is the most important thing and tells the computer where in the 4-stroke cycle (HIGHLY recommended reading if you don’t know about the 4 strokes and engine goes through) the engine is and therefore WHEN to inject the fuel. Typically this is some form of toothed wheel that is on the end of the crankshaft with a VR or Hall effect sensor that pulses each time a tooth goes past it. The more teeth the wheel has, the more precisely the computer knows where the engine is (Assuming it can keep up with all the pulses). By itself the toothed wheel is not enough however as the computer needs a reference point to say when the cycle is beginning. This is typically done by either a 2nd sensor that only pulses once every 4 strokes, or by using what’s know as a missing tooth wheel, which is the approach we have taken. This works by having a wheel that would ordinarily have, say, 36 teeth, but has then had one of them removed. This creates an obvious gap in the series of pulses which the computer can use as a reference once it is told where in the cycle the missing tooth appears. The photos below show the standard Cortina crankshaft end and the wheel we made to fit onto the end
To read the teeth, we initially fitted a VR sensor, which sat about 0.75mm from the teeth, however due to issues with that item, we ended up replacing it with a Hall Effect unit.
- Some way of knowing how much air is being pulled into the engine so that it knows HOW MUCH fuel to inject. In earlier fuel injection systems this was done with a Manifold Air Flow (MAF) sensor, a device which heated a wire that was in the path of the incoming air. By measuring the drop in temperature of the wire, the amount of air flowing over it could be determined (Although guessed is probably a better word as most of these systems tended to be fairly inaccurate). More recently systems (From the late 90’s onwards) have used Manifold Absolute Pressure (MAP) sensors to determine the amount of air coming in. Computationally these are more complex as there are a lot more variables that need to be known by the computer, but they tend to be much more accurate for the purposes of fuel injection. Nearly all aftermarket computers now use MAP and given how easy it is the setup (just a single vacuum hose going from the manifold to the ECU) this is the approach we took.
The above are the bare minimum inputs required for a computer to control the injection, however typically more sensors are needed in order to make the system operate smoothly. We used:
- Temperature sensors: As the density of air changes with temperature, the ECU needs to know how hot or cold the incoming air is. It also needs to know the temperature of the water in the coolant system to know whether it is running too hot or cold so it can make adjustments as needed.
- Throttle position sensor: The ratio of fuel to air is primarily controlled by the MAf or MAP sensor described above, however as changes in these sensors are not instantaneous, the ECU needs to know when the accelerator is pressed so it can add more fuel for the car to be able to accelerate quickly. These sensors are typically just a variable resistor fitted to the accelerator.
- Camshaft sensor: I’ll avoid getting too technical here, but injection can essentially work in 2 modes, batched or sequential. In the 4 strokes an engine goes through, the crankshaft will rotate through 720 degrees (ie 2 turns). With just a crank sensor, the ECU can only know where the shaft is in the 0-360 degree range. To overcome this, most fuel injection systems up to about the year 2000 ran in ‘batched’ mode, meaning that the fuel injectors would fire in pairs, twice (or more) per 720 degrees. This is fine and cars can run very smoothly in this mode, however it means that after being injected, some fuel mixture sits in the intake manifold before being sucked into the chamber. During this time, the mixture starts to condense back into a liquid which does not burn as efficiently, leading to higher emissions and fuel consumption. To improve the situation, car manufacturers starting moving to sequential injection, meaning that the fuel is only ever injected at the time it can go straight into the combustion chamber. To do this, the ECU needs to know where in 720 degrees the engine is rather than just in 360 degrees. As the camshaft in a car runs at half the crankshaft speed, all you need to do this is place a similar sensor on this that produces 1 pulse every revolution (The equivalent of 1 pulse every 2 turns of the crank). In our case, we had decided that to remove the distributor (which is driven off the crank) and converted it to provide this pulse. I’ll provide a picture of this shortly, but it uses a single tooth that passes through a ‘vane’ type hall effect sensor, so that the signal goes high when the tooth enters the sensor and low when it leaves.
- Oxygen sensor (O2) – In order to give some feedback to the ECU about how the engine is actually running, most cars these days run a sensor in the exhaust system to determine how much of the fuel going in is actually being burned. Up until very recently, virtually all of these sensors were what is known as narrowband, which in short means that they can determine whether the fuel/air mix is too lean (Not enough fuel) or too rich (Too much fuel), but not actually by how much. The upshot of this is that you can only ever know EXACTLY what the fuel/air mixture is when it switches from one state to the other. To overcome this problem, there is a different version of the sensor, known as wideband, that (within a certain range) can determine exactly how rich or lean the mixture is. If you ever feel like giving yourself a headache, take a read through http://www.megamanual.com/PWC/LSU4.htm which is the theory behind these sensors. They are complicated! Thankfully despite all the complication, they are fairly easy to use and allow much easier and quicker tuning once the ECU is up and running.
So with all of the above, pretty much the complete electronics system is covered. Of course, this doesn’t even start to cover off the wiring, fusing, relaying etc that has to go into making all of it work in the terribly noisy environment of a car, but that’s all the boring stuffECU
Finally the part tying everything together is the ECU (Engine Control Unit) itself. There are many different types of programmable ECUs available and they vary significantly in both features and price, ranging from about $400 to well over $10,000. Unsurprisingly there’s been a lot of interest in this area from enthusiasts looking to make their own and despite there having been a few of these to actually make it to market, the most successfully has almost certainly been Megasquirt. when we started with this project we had planned on using the 2nd generation Megasquirt which, whilst not having some of the capabilities of the top end systems, provided some great bang for bang. As we went along though, it became apparent that the Megasquirt 3 would be coming out at about the right time for us and so I decided to go with one of them instead. I happened to fluke one of the first 100 units to be produced and so we had it in our hands fairly quickly.
Let me just say that this is an AMAZING little box. From what I can see it has virtually all the capabilities of the (considerably) more expensive commercial units including first class tuning software (Multi platform, Win, OSX, linux) and a very active developer community. Combined with the Extra I/O board (MS3X) the box can do full sequential injection and ignition (With direct coil driving of ‘smart’ coils), launch control, traction control, ‘auto’ tuning in software, generic I/O of basically any function you can think of (including PID closed loop control), full logging via onboard SD slot and has a built in USB adaptor to boot!
In the next post I’ll go through the hardware and setup we used to make all this happen. I’ll also run through the ignition system that we switched over to ECU control.
Despite risking the occasional dirty look from a certain type of linux/FOSS supporter, I quite happily run the (currently) non-free Skype client on my HTPC. I have a webcam sitting on top of the TV and works flawlessly for holding video chats with family and friends.
The problem I initially faced however, was that my HTPC is 100% controlled by a keyboard only. Unlike Windows, the linux version of Skype has no built in shortcut keys (user defined or otherwise) for basic tasks such as answering and terminating calls. This makes it virtually impossible to use out of the box. On the upside though, the client does have an API and some wonderful person out there has created a python interface layer for it, aptly named, Skype4Py.
A little while ago when I still had free time on weekends, I sat down and quickly hacked together a python script for answering and terminating calls, as well as maximising video chats from the command line. I then setup a few global shortcut keys within xfce to run the script with the appropriate arguments.
I won’t go into the nitty gritty of the script as it really is a hack in some places (Particularly the video maximising), but I thought I’d post it up in case it is of use to anyone else. I’ve called it mythSkype, simply because the primary function of the machine I run it on is MythTV, but it has no dependencies on Myth at all.
The depedencies are:
- Python – Tested with 2.6, though 2.5 should work
- Skype4Py – any version
- xdotool – Only required for video maximising
To get the video maximising to work you’ll need to edit the file and set the screen_width and screen_height variables to match your resolution.
Make sure you have Skype running, then simply execute one of the following:
- ./mythSkype -a (Answer any ringing calls)
- ./mythSkype -e (End active calls)
- ./mythSkype -m (Maximise the current video)
The first time you run the script, you will get a prompt from Skype asking if you are happy with Skype4Py having access to the application. Obviously you must give your ascent or nothing will work.
Its nothing fancy, but I hope its of use to others out there.
A few months ago I switched from using the standard mythfrontend to Boxee, a web enhanced version of the popular XBMC project. Now Boxee has a LOT of potential and the upcoming Boxee Box looks very promising, but its fantastic web capabilities are let down here in Australia as things such as Hulu and Netflix streaming are not available here.
What we do have though is a national broadcaster with a reasonably good online facility. The ABCs iView has been around for some time and has a really great selection of current programs available on it. Sounds like the perfect candidate for a Boxee app to me.
So with the help of Andy Botting and using Jeremy’s Vissers Python iView download app for initial guidance, I put together a Boxee app for watching iView programs fullscreen. For those wishing to try it out, just add the following repository within Boxee:
Its mostly feature complete although there are a few things that still need to be added. If you have any suggestions or find a bug either leave a comment or put in a ticket at http://code.google.com/p/xbmc-boxee-iview/issues/list (A Google code site by Andy that I am storing this project at)
So that’s the short version of the story. Along the way however there has been a few hiccups and I want to say something about what the ABC (and more recently the BBC) have done to ‘protect’ their content.
The ABC have enabled a function called SWF Verification on their RTMP stream. This is something that Adobe offer on top of their RTMP products despite the fact that they omitted it from the public RTMP spec. That wouldn’t be so bad, except that they are now going after open source products that implement this, threatening them with cease and desists. Going slightly technical for a minute, SWF Verification is NOT a real protection system. It does not encrypt the content being sent nor does it actually prevent people copying it. The system works by requesting a ‘ping’ every 60-90 seconds. If the player can’t provide the correct response (Which is made up of things such as the date and time and the phrase “Genuine Adobe Flash Player 001″) then the server stops the streaming. Hardly high tech stuff.
In my opinion the ABC has made a huge mistake in enabling this as it achieves nothing in stopping piracy or restricting the service to a certain platform and serves only to annoy and frustrate the audience. There is a patch available at http://code.google.com/p/xbmc-boxee-iview that allows Boxee to read these streams directly, however until such time as this is included in Boxee/XBMC mainline (Watch this space: http://trac.xbmc.org/ticket/8971) or the ABC come to their senses as disable this anti-feature, this Boxee app will use the flash interface instead (boo!)
So that’s it. Hope the app is useful to people and, as stated above, if there are any problems, please let me know.
I should’ve mentioned this originally, Andy and I did actually contact the iView team at the ABC regarding SWF verification. They responded with the following:
Thanks for contacting us re-Boxee. We agree that it’s a great platform and ultimately appropriate for iView iteration. Currently we’re working out our priorities this year for our small iView team, in terms of extended content offerings, potential platforms and general enhancements to the site.
Just some background on our security settings. We have content agreements with various content owners (individuals, production companies, US TV networks etc) a number require additional security, such as SWF hashing. Our content owners also consider non-ABC rendering of that content as not in the spirit of those agreements.
We appreciate the effort you have put into the plug-in and your general interest in all things iView. So once we are on our way with our development schedule for “out of the browser” iView, we’ll hopefully be in a position to share our priorities a little more. We would like to keep in touch with you this year and if you have any questions or comments my direct email is ********@abc.net.au.
The app is currently in a WORKING state. If you are experiencing any problems, please send me a copy of your Boxee log file and I will investigate the issue.
6th Floor, 200 Victoria St. Carlton VIC 3053Link: http://luv.asn.au/meetings/map
- Russell Coker, M.2
- Rodney Brown, CRCs
Late arrivals, please call (0490) 049 589 for access to the venue.
Before and/or after each meeting those who are interested are welcome to join other members for dinner. We are open to suggestions for a good place to eat near our venue. Maria's on Peel Street in North Melbourne is currently the most popular place to eat after meetings.
Linux Users of Victoria Inc. is an incorporated association, registration number A0040056C.August 2, 2016 - 18:30
Infoxchange, 33 Elizabeth St. RichmondLink: http://luv.asn.au/meetings/map
This hands-on presentation and tutorial with Wen Lin will introduce the various types of file sharing in Linux - from the more traditional NFS & Samba to the newer cloud-based ones like Dropbox, Google Drive and OwnCloud. The primary audience of the talk will be for beginners (newbies), but hopefully some of you who are familiar with Linux will get something out of it as well.
The meeting will be held at Infoxchange, 33 Elizabeth St. Richmond 3121 (enter via the garage on Jonas St.)
Late arrivals, please call (0490) 049 589 for access to the venue.
LUV would like to acknowledge Infoxchange for the venue.
Linux Users of Victoria Inc., is an incorporated association, registration number A0040056C.August 20, 2016 - 12:30