Fedora Linux Support Community & Resources Center
  #1  
Old 6th April 2017, 03:05 PM
ocratato Offline
Registered User
 
Join Date: Oct 2010
Location: Canberra
Posts: 2,650
linuxfirefox
A journey into UPnP

I thought I might relate my adventures in the wonderful world of DLNA and UPnP.

I have a Panasonic DVR that I mostly use to time-shift various free-to-air TV programs - the very few that are worth watching. Occasionally there will be something that I feel might be worth keeping a bit longer in my video file collection.

My technique used to involve trimming out the ads and other noise and then writing the video to a re-writable DVD. This could then be ripped into my collection. This was actually a fairly convenient process - the write to DVD could be done while I slept, and when I had a few I could use Handbrake to extract the video files.

This all came to stop when the DVR's DVD drive shredded its position servo gears, and there seemed to be very little point in getting it repaired. Since it had a network connection I thought of trying to get the files from it using its DLNA interface.

So, I go buy a cat5 cable and a connector so that I don't have to get behind the hifi system but can still plug in the long cat5 cable. Once connected to the network the DVR spends ages updating its firmware - frustrating. Eventually I was able to get to its screen for setting up DLNA. It lists the MAC address of potential clients - an empty list

Next I start Googling for suitable client programs. The first was eezUPnP, but this seemed to be just for audio. Next was XBMC. This has the most awful interface I have ever attempted to use - it might be OK when used on a TV, but on a laptop - With XBMC running it couldn't see any servers, and the DVR still couldn't see any clients. I suspected a firewall issue - but what port? More Googling gave no help, so I fired up lsof to see what XBMC was trying to use. It seemed to have some interest in several so I opened them in the firewall. Finally a glimmer of success - the DVR now listed my laptop as a potential client, which I then confirmed. XBMC could now see the DVR and even play a video. I know now that udp 1900 is the important one to open on the client.

Unfortunately, to save the video in XBMC you need a plug-in and my version of XBMC from the SL7 repos was apparently too old to get one that was compatible. So, next I uninstalled XBMC and installed VLC, and happily it was able to both play and save the video files.

At this stage I was both happy that I could get at the files, and sad that it was such a long and frustrating journey.

The next problem is that processing the files one-at-a-time using the VLC GUI was going to get tedious rather quickly - I needed some way of using ffmpeg to read the files in a batch script. Fortunately there is a program called djmount that uses libfuse to create a virtual file system of the DVR. Just the thing for doing batches.

So I download the source for djmount and successfully build it. Of course it fails to recognise my DVR

Checking the logs from running it in debug mode I see that my DVR is reporting version 2 of some standard, but djmount was only accepting version 1. Just for fun, I simply changed the string, and surprisingly it now recognised the DVR, but then failed trying to allocate some memory. At first I assumed the usual problem of corrupted memory causing malloc to fail, but eventually noticed it was being asked for a stupidly large amount of memory.

My system is 64-bit, but djmount seems to have been written for 32-bit. While it compiles, there were lots of places where 32-bit and 64-bit numbers were incorrectly used. The best example was in a function that used varargs parameters but passed one in as 32-bit, but extracted it as 64-bit. This resulted in a huge number and messed up all the following parameters. Fixing all these problems had the side effect that djmount was now able to handle files of more than 2GB which, according to some forums, is currently not possible.

The program was now able to get a directory listing from the server, but only 36 of the 586 files were shown. My server was only returning 12 at a time, not the entire list as djmount expects. It does a couple of retries and then gives up. I simply changed it to 50 retries and got the entire directory list - over and over - it was in some sort of loop. More debugging and eventually tracked it down to the cache timeout. This interval was just a bit less than the time it took to get the directory list. Changing the timeouts from a few seconds to an hour or so (which should be reasonable for a DVR) fixed that issue.

Now it was time to copy a file. It started off OK, but after a while it reported yet another error and stopped the transfer. I traced the problem to the function that reads from the socket and parses the results. The parsing was sort of expecting that the socket read might not return a full buffer, but the parser never checks to ensure its got enough to work with. This brain dead code has Intel copyrights all over it - something that makes me quite sad. I rewrote the socket read so that it doesn't return until its got as much data as possible - hence the parser can work properly.

There is an interesting issue with files copied from the DVR - they have snippets of the video that was edited out. My assumption is that the DVR stores the files in a format that is not accurately editable (most video formats have this problem) and keeps some metadata containing the frame numbers to be skipped on replay. These frames are skipped when playing back the video, and also when preparing to write a DVD but not when playing via UPnP. This means that eventually I will need re-edit the files, but that is a task for another time.

Finally, today I created a script to get batches of files and process them with ffmpeg. It has code to handle stopping and restarting.

My last task is to clean up my debugging code and make a patch for the changes.
__________________
Has anyone seriously considered that it might be turtles all the way down?
That's very old fashioned thinking.
The current model is that it's holographic nested virtualities of turtles, all the way down.
Reply With Quote
  #2  
Old 6th April 2017, 03:46 PM
HaydnH's Avatar
HaydnH Offline
Registered User
 
Join Date: Feb 2005
Location: London, UK
Posts: 509
windows_7chrome
Re: A journey into UPnP

Wow that sounds complicated! I personally use mediatomb as a music/video server sitting on a NAS box, a cheap Chinese brand (Hi-Media) hardware DLNA renderer plugged in to the TV/stereo/Wifi and bubbleUPNP on my phone. As long as the NAS and renderer are turned on then I select a song on my phone and the renderer grabs it from the NAS box to play on the TV/stereo. Fairly easy.
Reply With Quote
  #3  
Old 6th April 2017, 04:04 PM
PabloTwo's Avatar
PabloTwo Offline
"Registered User" T-Shirt Winner
 
Join Date: Mar 2007
Location: Seville, FL
Posts: 7,562
linuxchrome
Re: A journey into UPnP

Hmmm, sounds more like you were more consumed with "cracking this nut" rather than actually wanting to watch those locked away videos on the broken DVR. In any case, that was one heck of a project.
Reply With Quote
  #4  
Old 7th April 2017, 01:45 AM
ocratato Offline
Registered User
 
Join Date: Oct 2010
Location: Canberra
Posts: 2,650
linuxfirefox
Re: A journey into UPnP

Quote:
Originally Posted by PabloTwo View Post
Hmmm, sounds more like you were more consumed with "cracking this nut" rather than actually wanting to watch those locked away videos on the broken DVR. In any case, that was one heck of a project.
Its hard to stop when it appears that "just one more fix will get it going".
__________________
Has anyone seriously considered that it might be turtles all the way down?
That's very old fashioned thinking.
The current model is that it's holographic nested virtualities of turtles, all the way down.
Reply With Quote
  #5  
Old 7th April 2017, 02:41 PM
beaker_'s Avatar
beaker_ Offline
Registered User
 
Join Date: Nov 2008
Location: Canada
Posts: 2,718
unknownsafari
Re: A journey into UPnP

Sounds like a lot of effort to avoid using mythtv.
Reply With Quote
  #6  
Old 7th April 2017, 03:03 PM
ocratato Offline
Registered User
 
Join Date: Oct 2010
Location: Canberra
Posts: 2,650
linuxfirefox
Re: A journey into UPnP

Quote:
Originally Posted by beaker_ View Post
Sounds like a lot of effort to avoid using mythtv.
Not really. Buying a commercial DVR was a lot simpler than trying to set up MythTV - especially at the time I bought it - many years ago.

One of thereasons for not getting it repaired is because my next video project may well involve MythTV, but I think that will be quite some time in the future. For now I just need to make a backup of the DVR before it develops some other problem.
__________________
Has anyone seriously considered that it might be turtles all the way down?
That's very old fashioned thinking.
The current model is that it's holographic nested virtualities of turtles, all the way down.
Reply With Quote
  #7  
Old 7th April 2017, 05:37 PM
beaker_'s Avatar
beaker_ Offline
Registered User
 
Join Date: Nov 2008
Location: Canada
Posts: 2,718
linuxfirefox
Re: A journey into UPnP

For when you do, these probably are the easiest. Especially if need multiple tuners and like to stream (rtp). No FM radio though.

https://www.thinkpenguin.com/gnu-lin...-canada-europe
Reply With Quote
Reply

Tags
journey , upnp

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
A journey through file contexts. sepoto Programming & Packaging 5 14th February 2011 08:13 AM
UPnP - on or off? Nighthawk4 Security and Privacy 3 4th April 2005 05:32 PM


Current GMT-time: 08:02 (Sunday, 22-10-2017)

TopSubscribe to XML RSS for all Threads in all ForumsFedoraForumDotOrg Archive
logo

All trademarks, and forum posts in this site are property of their respective owner(s).
FedoraForum.org is privately owned and is not directly sponsored by the Fedora Project or Red Hat, Inc.

Privacy Policy | Term of Use | Posting Guidelines | Archive | Contact Us | Founding Members

Powered by vBulletin® Copyright ©2000 - 2012, vBulletin Solutions, Inc.

FedoraForum is Powered by RedHat