Removing dead tracks from your Banshee library

October 6th, 2013

I used MusicBrainz Picard to reorganize hundreds of the files in my music collection. For some reason Banshee spotted that new files appeared, but it didn’t remove the old ones from the library. My library had hundreds of entries relating to files that no longer existed on disk.

I would have hoped that Banshee would just skip when it got to a playlist entry that didn’t exist, but unfortunately (as of v2.4.1 in Ubuntu 12.04) it decides to stop playing at that point. I have to switch to it and double click on the next playlist entry, and even then it thinks it is playing the file that it tried to play before. So I have to double click again.

I have filed a bug about this.

In the mean time I wanted to remove all the dead entries from my library. I could delete my library and re-import it all, but I wanted to keep some of the metadata. I knew it was an SQLite database, so I poked around a bit and came up with this. It seems to work, so maybe it will be useful for someone else.

Wanted: cheap but cheerful small Linux device

September 3rd, 2013

I changed ISP recently for my broadband at home and switched from ADSL2+ to FTTC, so that’s required a new broadband router.

Initially I got things working with the Technicolor TG582N as supplied by the ISP, but it appears quite horrible in most of its functionality. I find most cheap domestic broadband routers are, to be honest. Little plastic blobs with the absolute minimum spec of hardware, configured via web interfaces that can politely be described as clunky, and packing many unwanted features.

With FTTC here in the UK you have a separate NTE box supplied by British Telecom and then you supply (or your ISP supplies) a router that connects to that by Ethernet and talks PPP-over-Ethernet to your ISP. So, anything that can do PPPoE works as the router, no special hardware required. Any Linux box will do.

I had this Soekris net4801 box that I purchased in 2005, been running it constantly ever since, and it still works fine. It’s a nice little thing; 266MHz fanless CPU, 128MiB RAM, three 10/100 Ethernet ports and CompactFlash for storage. Draws under 10W when idle and not a lot more at full tilt.

Really quite expensive though. After delivery charges, purchase of compatible PSU and CF card and currency conversions are done you’re probably talking £200 now and I seem to recall it was similar back in 2005 too.

I upgraded that from Debian etch to lenny to squeeze to wheezy — which went remarkably without incident by the way, a testament to Debian’s excellent upgrade procedure — and set it to work as the router. Since it’s just a relatively conventional Debian install it’s really easy to configure PPPoE, IPv4, NAT, IPv6, firewalling and anything else.

There’s a couple of things I’m not too happy about though.

What if it dies?

If you have a Soekris last several years then it’s going to be pretty reliable. There’s no moving parts, the most likely faults are going to be the CF card or the power supply. Even so, this one’s been in service about 8 years and that’s a really good innings. It could go any time and then what will I replace it with?

Of course I still have the Technicolor and that will work well enough to get connectivity until I put something better in its place again, but what would be that better thing?

Back in 2005 I had a bit more disposable income than I do now and £200 was okay to spend on something I was interested in playing with. I’m done playing with it now though and spending £200 to end up with a Linux box that runs at 266MHz and has 128M RAM is going to hurt. Also the net4801 is end of life so will get harder and harder to purchase new, and any replacement will cost a little more.

Is the Soekris really beefy enough?

Right now I only have 40M down, 10M up FTTC and the Soekris doesn’t appear to be limiting that any more than the Technicolor limited it.

Conceivably though I may one day upgrade it to 80/20 or more and that is starting to push the limits of a 100M Ethernet port, let alone a 266MHz CPU.

As you would expect from a 266MHz CPU with 128M RAM it’s dog slow at doing anything much in user land. This is a pretty minor gripe as the use case here is that of an appliance, like the broadband router it replaced. You shouldn’t really need to touch it much. Something slightly less puny would be a nice bonus though.

Options

HP Microserver

HP have been doing cash back deals on their Microserver range for a few years now. I already have one here at home being a file server and a few other bits and pieces. If they were still doing the cash back then I’d strongly consider buying another one to use for this.

It would draw a fair bit more power than the Soekris does, but they are still quite efficient machines and I would probably find it more things to do since it would be a lot more capable.

Without the cash back though I don’t think it can be justified. Retail price of a Microserver at the moment is around £265+VAT.

Update: It appears the cash back offer has returned, at least for September 2013!

http://www.serversplus.com/servers/tower_servers/hp_tower_servers/704941-421

Some Linksys WRT device with OpenWrt

It’s a contender, but it will leave me with some cheap nasty hardware running a non-standard Linux distribution on an ARM CPU. I’m sure OpenWrt is great but I don’t know it, I’d have to learn it just for this, and it’s not likely to be useful knowledge for anything else.

If possible I want to remain running Debian.

More enterprisey router hardware from Cisco or Mikrotik

This would certainly work; a Cisco off of ebay may be cheap enough, otherwise a new Mikrotik Routerboard would be within budget. Say an RB450G.

The main issue again would be it’s not Linux. That’s not necessarily a bad thing, it’s just that it wouldn’t feel familiar to me. I know how to configure everything in Linux.

Something from Fabiatech

I stumbled across a blog post by Richard Kettlewell entitled Linux In A Small box. In it he considers much the same issue as I have been, and ends up going for a Fabiatech FX5624

Looks good. £289+VAT though.

omg!! Raspberry Pi everywareeeeeeeeeeeeeeeeeee!!!!!

Yeah, Raspberry Pis are nice pieces of kit for what they are designed for. Which is not passing large amounts of network traffic. They only have one 100M Ethernet, and it’s driven by USB 2.0 so it’s going to suck. It will suck even more when you attach a USB hub and more USB Ethernets.

Something from Jetway

Alex suggested looking at these devices. They look quite fun.

A bare bones system that on paper should do the job (1.6GHz Intel Cedar Trail CPU, two Realtek gigabit Ethernet, one SO-DIMM slot for up to 4GB RAM) seems to be £149+VAT.

There seems to be a good selection of other main boards and daughter boards if that config wasn’t suitable.

Anyone got any personal experience of this hardware?

This Is Not An Exit

I still don’t know what I will do. I might put off the decision until the Soekris releases its magic blue smoke. I would be interested to hear any suggestions that I haven’t thought of.

Here are the requirements:

  • Capable of running a mainstream Linux distribution in a supportable fashion without much hacking around.
  • Has at least two gigabit Ethernet ports.
  • Is beefier than a 266MHz Geode CPU with 128M RAM
  • Easy to run its storage from an inexpensive yet reasonably reliable medium like CompactFlash or SD/microSD. Write endurance doesn’t really matter. I will mount it read-only if necessary.

Some nice-to-haves:

  • At least one serial port so I can manage it from another computer when its network is down, without having to attach a VGA monitor and keyboard. The Soekris manages this perfectly, because it’s what it’s designed for. It doesn’t even have a VGA port.
  • Total configuration of the BIOS from the serial port, so a VGA monitor and keyboard are never necessary. Again, that’s how Soekris products work.
  • Ethernet chipsets that are actually any good, i.e. not Realtek or Broadcom.
  • Capable of being PXE booted so that I don’t have to put the storage into another machine to write the operating system onto it.

Get with the programme

July 9th, 2013

Apparently my lack of any particular interest in the following things is like farting in nerd church:

  • Doctor Who
  • Firefly
  • Amanda Palmer
  • Brian Blessed

Nothing personal against any of these people/things. I just don’t really “get” them.

“My IP is blocked by a repressive regime, can I have a different one?”

July 7th, 2013

I asked this question on Twitter yesterday and got a wider range of responses than I expected, although from a limited number of people. So I wondered what others would think.

Say you sell virtual machines and a customer says:

My service allows journalists and others inside repressive regimes to get their stories out. My IP address is being blocked by one of these repressive regimes. Can you switch it for another one?

Would you grant that request?

Assume you have never heard of their service or anyone that uses it, have no independent verification of what whether they are saying is true, and haven’t yet looked for any.

Responses so far could roughly be grouped as:

  • 2x “Yes; it’s a reasonable request and other networks’ policies are their own business”
  • 2x “Yes; once, but check it’s not some global spam blacklisting issue”
  • 3x “Yes; but charge them for your time each time they ask for this”
  • 2x “No; you’ll end up with all your IPs blocked, which may affect other customers”
  • 1x “No; tell them to use a cloud with a constantly-changing IP address” (involves me losing the customer)

What would you do?

If you think suicide is weak or shameful, you just don’t understand

January 24th, 2013

Recently, someone fairly well known in certain circles committed suicide. People commit suicide all the time of course, but this person could fairly be described as a form of genius, a polymath, hero to many. Since their sphere of influence was (is!) strongly Internet-based, the net has been alive for weeks with people feeling the need to comment upon it.

I haven’t made a comment upon it because I didn’t know this person. I knew of them, of many of their great works and deeds and philosophies. Didn’t agree with some of them, but there you go. Anyone who knew anything about this person knows that the world is much worse off to not have them in it, so no one needs to hear that from me. Speaking about their circumstances specifically though is something I still don’t feel comfortable about. It feels to me a bit too much like some sort of leveraging of grief in order to just make statements about oneself.

Yes, I do realise that by just saying that stuff I have passed comment and now you all know something of my politics and beliefs so I’m really cool for how much I care right? Well, I couldn’t avoid it as otherwise it ends up coming across like, “I don’t care that they died; I didn’t know them.”

This particular incident though, being fodder for Internet discussion by persons not closely emotionally tied to the deceased, has lead to me now seeing quite a few people expressing views like, “don’t hero-worship someone who killed themselves, they’re weak and selfish.” Or, “I’m so disappointed in them that they felt this was necessary.” Like, publicly expressing them, for the world to see. Some influential people.

I still, weeks later, can’t quite put into words how much I am disgusted with these sorts of comments, or even exactly why I am. This is my best attempt so far and it’s not really going very well is it?

This is not about the individual concerned; these are views that some people express whenever there is a suicide that is notable enough to be a topic of discussion, but emotionally remote enough from them that they feel able to “speak their minds.”

There is just some shocking level of arrogance involved when you say that someone was weak, selfish, acted shamefully, disappointed you (YOU, for fuck’s sake!) by ending their own life.

I don’t entirely (thankfully) know what goes through someone’s mind when they decide to end it all but I am pretty sure that they are in such a bad place that any thought of what other people will think has long ago ceased to have any positive effect and probably has the opposite instead.

I don’t know how to stop people killing themselves through despair. I don’t know what the best strategies are. But please just stop acting like suicidal people feel they have some sort of choice, that if they would just not let everyone down so much it would go better for them. I can’t begin to imagine that helps.

Their action must have come from a place where they truly believe no choice exists, and if you can’t sympathise with that then please at least maintain a respectful silence.

Scanning for open recursive DNS resolvers

January 11th, 2013

A few days ago we unfortunately had some abuse reports regarding customers with DNS resolvers being abused in order to participate in a distributed denial of service attack.

Amongst other issues, DNS servers which are misconfigured to allow arbitrary hosts to do recursive queries through them can be used by attackers to launch an amplified attack on a forged source address.

I try to scan our address space reasonably often but I must admit I hadn’t done so for some time. I kicked off another scan and found one more customer with a misconfigured resolver, which has since been fixed.

After mentioning that I would do a scan I was asked how I do that.

I use a Perl script I’ve hacked together over the last couple of years. I took a few minutes to tidy it up and add a small amount of documentation (run it with --man to read that), so here it is in case anyone finds it useful:

Update: This code has now been moved to GitHub. If you have any comments, problems or improvements please do submit them as an issue and I will get to it much quicker. The gist below is now out of date so please don’t use it.

Using the default 100 concurrent queries it scans a /21 in about 80 seconds (YMMV depending upon how many hosts you have that firewall 53/UDP). That scales sort of linearly with how many you do, so using -q 200 for example will cut that down to about 40 seconds. It’s only a select loop though so it’ll use more CPU if you do that.

Two things I’ve noticed since:

  • It doesn’t handle failing to create a socket with bgsend so for example if you run up against your limit of file descriptors (commonly ~1024 on Linux) the whole thing will get stuck at 100% CPU.
  • One person reporting a similar situation (bgsend fails, stuck at 100% CPU) when they allowed it to try to send to a broadcast address. I haven’t been ale to replicate that one yet.

Converting an IPv6 address to its reverse zone in Perl

November 29th, 2012

I’m needing to work out the IPv6 reverse zone for a given IPv6 CIDR prefix, that is a prefix with number of bits in the network on the end after a forward slash. e.g.:

  • 2001:ba8:1f1:f004::/64 → 4.0.0.f.1.f.1.0.8.a.b.0.1.0.0.2.ip6.arpa
  • 4:2::/32 → 2.0.0.0.4.0.0.0.ip6.arpa
  • 2001:ba8:1f1:400::/56 → 0.0.4.0.1.f.1.0.8.a.b.0.1.0.0.2.ip6.arpa

I had a quick look for a module that does it, but couldn’t find one, so I hacked this subroutine together:

Is there a more elegant way? Is there a module I can replace this with?

Must support:

  • Arbitrary prefix length
  • Use of ‘::’ anywhere legal in the address

How do I send a message of praise to a tube driver’s manager?

August 29th, 2012

Earlier today I was helping Mum to the station after she’d been down to stay with us for a few days. We were on the Piccadilly line, and I started to notice that the driver was doing a lot of talking. Apart from his upbeat and friendly tone—itself sadly a rarity it seems—he had something useful to say before each stop.

He was telling us things like:

  • Which side of the train the doors would open.
  • Where the elevators were located on the platforms and how many people could fit in them (“This elevator has a capacity of fifteen, one five, persons”!)
  • Short cuts for interchange between lines (“you’ll find it quicker to go up to the ticket hall but then do a U-turn and go back down to the Jubilee line”)
  • Between which stations it would be possible to make a mobile phone call.

Some of his attempts at humour may not have been to everyone’s taste…

Cross the platform for the District line to Barking. That’s Barking, woof woof!

…but looking down the carriage I saw more than one person with a smile on their face. On a dreary London afternoon that’s got to be a win.

As we started to reach Central London his comments became more like mini tourist information, calling out the interesting places that are near each station and I heard at least one person comment, “Oh I didn’t know that was there!”

I’ve lived in London since 2004 and I’ve never heard a tube driver put so much effort and personality into their announcements. It was a really welcome surprise; too often you can hear the exasperation in the drivers’ words and they don’t even say “please.” It’s worse than leaving it to the automated announcements.

No one loves their job all day every day and you can’t manufacture sincere enthusiasm, but it makes so much difference. Naturally we primarily want the trains to be punctual and reliable, but once that is covered, having an actually pleasant personality when dealing with the public also goes a long way.

So I was thinking, Transport for London employees probably just get a lot of grief when things go wrong, and a lot of the time that will be entirely out of their control anyway, but still they have to be the interface with the public and deal with it. Here is a chap that did personally go out of his way to do a good job; someone should say thanks. Maybe he’ll keep doing it. Maybe he’ll get others to do it.

I’d like to say thanks to him for making our journey entertaining and for helping to make London a more appealing city for our visitors. How do I make sure his manager sees this?

It was a Piccadilly line train headed for Cockfosters. It arrived at Piccadilly Circus at 13:20 Wednesday 29th August.

Update: Looks like it was this guy:

http://districtdave.proboards.com/index.cgi?board=picc&action=display&thread=19882

Opinions are divided!

Personally I would take his cheery announcements every day twice a day rather than the norm. Don’t stop!

50 Shades of grep (NSFW)

July 9th, 2012

<grifferz> unixmen.com is a bit of an odd name isn’t it

<taras> i thought it was going to be unix fanfic

<grifferz> maybe you could write and post some there

<taras> Torvalds cupped Stallman’s bristly sack in his delicate Finnish hand

<taras> “Let’s see if you’ve ‘Hurd’ of this position,” he whispered

<grifferz> 50 Shades of grep

Strategies for talking to Labour MPs about the draft Communications Data Bill

July 2nd, 2012

Dear Lazyweb,

I’m thinking of having a conversation, face-to-face, with my MP about the draft Communications Data Bill. I’ve already done some research on the logical and moral reasons why the bill is a terrible idea. I feel pretty confident in how I can articulate those points.

My MP is a Labour MP though, so I am wondering what additional points I can bring up that will appeal to the Labour party. I’m hoping that those of you who aren’t going to write to or visit your MPs would instead be willing to lazyweb me some advice.

Most obviously there is the fact that Labour is in opposition so anything they can say to criticise the coalition government is a good thing for them.

On that score, we have the following ace in the sleeve:

Labour have subjected Britain’s historic freedoms to unprecedented attack. They have trampled on liberties and, in their place, compiled huge databases to track the activities of millions of perfectly innocent people, giving public bodies extraordinary powers to intervene in the way we live our lives. The impact of this has been profound and far-reaching. Trust has been replaced by suspicion. The database state is a poor substitute for the human judgement essential to the delivery of public services. Worse than that, it gives people false comfort that an infallible central state is looking after their best interests. But the many scandals of lost data, leaked documents and database failures have put millions at risk. It is time for a new approach to protecting our liberty…

– David Cameron, “Invitation to Join the Government of Great Britain”, 2010

The problem I can foresee is, what if my MP points out that the bill is almost identical to the one Labour tried to introduce in 2008/2009 and therefore is something that she is broadly in favour of?

If this does happen then I could possibly try the angle that although Labour did investigate it, they found it had too much public opposition and was technically infeasible at a sensible cost. In effect I could try to spin this as a further attack point on the coalition that despite a past government having already spent time and money on this and getting an answer the coalition doesn’t like, they are going to do it all over again. If anything in an age when we have even less money and time to be spending on it.

I think my MP is a fan of Harriet Harman. Has Harriet Harman yet said anything that would indicate opposition of the CDB in all its past incarnations? What was Ms Harman’s stance on the CDB when Labour were proposing it? Give me links, people.

Any more ideas why Labour should get their hate on?

(If my MP is reading this, no disrespect is intended and I’m fully glad that you are able to read this without having to snarf it off of a black box on my ISP’s network! I look forward to having a chat with you about it.)

Update:

Just after writing this I contacted my MP’s secretary and asked for details of her surgeries. I was emailed back and asked if I would instead like to speak to her on the phone.

I was disappointed at not being offered a face to face meeting, but not wanting to appear to be a nutter I agreed to this and a call was scheduled for 2.30pm on Friday 6th July.

Depending on how it goes I may still try to have a follow up meeting. Whatever the case I will put all my concerns in writing anyway.

Update Friday 6th July:
It got to 3pm and I hadn’t had the call I was promised, so I emailed the MP’s secretary again to ask if it was still happening. The secretary replied immediately that she was sorry and would text my MP to find out what had happened.

Shortly afterwards I received a phone call from my MP, who was obviously in her car, apologising. She said that she had hoped to speak to me between meetings but was now having to rush to another and wouldn’t be able to, and asked if we could re-schedule the call for Monday 9th July.

I have agreed to that.

Update Monday 9th July:

Our phone call had been arranged for 11.30 today. By 12.30 I hadn’t received a call, so I emailed the MP’s secretary again to ask if it was still happening. As of 2pm I’ve received no response and I’m not available on the phone for the rest of today.

I’m rather disappointed that it seems to be so difficult to speak to my MP about this important piece of proposed legislation. I suspect that, as my first query about surgeries was turned into a suggestion of a phone call, that my MP doesn’t hold surgeries (I can find no details of any surgeries she may have held). I haven’t got infinite time to spend on this and am considering just putting my views in writing and calling an end to it.


Update Tuesday 10th July:

I received an email reply this morning from my MP’s secretary saying that she (the secretary) had not been working the day before so had not seen my email querying the lack of phone call. She asked me to confirm if a phone call had taken place. I replied that it had not taken place.

Around 5.45pm someone else from my MP’s office emailed me to ask if I was able to take a phone call on Friday (13th). No explanation of why the previously arranged call hadn’t happened.

I’m unsure at this point whether to suck it up and agree to reschedule the call, or whether to give up. If this were anyone in my personal or professional life I would have given up on them by now. But that doesn’t help anyone does it, and leaves me open to the criticism that I just didn’t try hard enough.

Update Wednesday 11th July:

Yesterday I was rather frustrated by the whole thing, but after a night’s sleep I’ve had chance to calm down and I’ve decided that in the interests of getting my point across I will swallow my pride.

I’ve again asked if a face to face conversation at a surgery is possible (because I think this is a complicated subject that isn’t best discussed on the phone), but if not then I’ve agreed to reschedule the call for Friday.

Around 09:45 I received a phone call from Mary Jo (secretary) who apologised unreservedly for how I had been treated and promised to sort it out today. I’m now glad that I didn’t lose my temper yesterday.

It’s since been arranged that I will have a face to face meeting with my MP on Friday 13th, so that’s great news.

Update Friday 13th July:

I think the meeting today went well. Ms Malhotra gave me plenty of time to discuss my concerns, seemed to genuinely take many of them on board and offered me some very useful advice for how I might like to take things further. She apologised for the initial problems I’d had in getting to speak to her. Overall I’m very glad that I persevered with this.

Of course there is much more to do, but making my views known to my MP was a necessary step.