I have two use cases for what could be defined as IPTV software. The first is the traditional case, I have a DVB-T card and want to record shows and pause live TV. The second is surveillance. I have a few IP MJPEG cameras and want to be able to review recordings and run motion detection to figure out where the interesting events are. In both cases there’s a continuous live stream of video that I may want to watch live or review parts of at a later date.

My first attempt at solving both these problems was with mythtv and zoneminder respectively. These are the traditional solutions in the linux world and they really show their age. Mythtv’s client-server model is based on all the clients fiddling with the same underlying mysql database under the hood. And while mythtv seems to be evolving zoneminder doesn’t seem to be under very active development.

The last few weeks I tried out some new options. I have now replaced mythtv with tvheadend and zoneminder with motion. Both of these are much better options for my needs. They don’t have as many features but they do the basics and they do them better. I’m happy with the swap, but the new tools are just better mouse traps, they fix my annoyances with the other two but don’t really give me the features I want.

The fundamental assumption of all these programs is that you should only save to disk video that you’ve already confirmed someone is interested in. tvheadend like mythtv will only start capturing a tv stream if it has been configured to record a particular show, or a client has requested live streaming. motion, like zoneminder, will only save a video file after it has detected motion. This way of working makes some very interesting use cases impossible:

• Turning on the TV in the middle of the news and deciding to rewind to the start
• Finding out an acquaintance was on a TV show earlier today and saving that recording for posterity
• Watching 1-2 hours of surveillance footage spanning a few important motion events.
• Treating your surveillance cameras as any other TV channel and pausing them and rewinding while watching live

It turns out all the interesting use cases I had in mind are not possible to do because these applications have already decided to throw away the data by the time you want to make a decision to look at it. Thinking about it today I came up with a design that solves the problem and supports both use cases. Here’s what the app would do:

1. Take the original stream (be it a DVB MPEG4 channel or MJPEG) and write it continuously to disk splitting it into consecutive files of a given chunk size.
2. If a client wants a live stream, just start streaming from disk at the latest chunk. Rewind works fine because he’s really being streamed a recording, and you can rewind as far back as the program has been saving the stream.
3. Because disk space isn’t infinite you set a limit to how far back you keep the chunks (say 5 days of the stream) and continuously delete older chunks.
4. If a user wants to save all the episodes of their favorite show, just mark the chunks where the show is (some may only partially be the show) to not be deleted by the continuous cleanup.
5. Motion detection is just a special case of the previous. The algorithm runs automatically over the live stream and whenever an event is detected the chunks where it’s video is in get marked for saving.

There’s a bunch of details about how to do this efficiently that I’ve been mulling over but are not really the point of this blog post. What I’m wondering is if there’s something I’m missing that would make this not feasible. The ones I can come up with are:

• Most people that use DVB solutions aren’t able to tune all channels at all time, because they’re on different frequencies. Even if they could the disk bandwidth might not be enough to save every channel at the same time. In my case this isn’t a problem as Portuguese DVB-T is 4 or 5 channels on a single frequency and DVB stream, they’re only logically separated into channels.
• The continuous writing to disk might shorten its lifetime substantially. This I’m particularly worried about and wondering if anyone has any data on disk longevity given different write loads. A mitigation strategy would be to dedicate a “dirty” disk to do the continuous saving and move chunks into “clean” disks only when a show is to be saved or a motion event to be captured. This way if the continuously thrashed disk fails you lose the ability to rewind into things you haven’t stored yet, but you don’t lose the important events/shows.

If I had the time right now this would be a great project to learn Go with. I’d try and write the basic server as small and concurrent compiled program and then code the bells and whistles in ruby. Anyone see some obvious reason why this kind of thing wouldn’t work?

(Written on 24 Feb 2013)