Posts mit dem Label mythtv werden angezeigt. Alle Posts anzeigen
Posts mit dem Label mythtv werden angezeigt. Alle Posts anzeigen

Freitag, 14. Juni 2013

Validating audio and subtitle track availability in collections

Archiving a full set of episodes of a TV-series has some challenges: missing episodes, bad quality, recording issues - and language and subtitles. You may have missed one episode on your preferred HDTV channel and got one in SD lacking subtitles and original language.

Below script (yes, I know it's ugly) will dump language information for audio tracks and subtitle tracks from a MKV file in a sorted order. This allows a quick and easy check if your archived episodes have all the same language information.

#!/usr/bin/perl

while (<>) {
last unless /\S/;
$lines .= $_;
}

@tracks = split /A track/, $lines;

@audiotracks = grep/type: audio/, @tracks;
@subtracks = grep /type: subtitles/, @tracks;

foreach (@audiotracks) {
push(@atracks, $1) if / Language: (\w{3})/;
}
foreach (@subtracks) {
push(@stracks, $1) if / Language: (\w{3})/;
}

print "Audiotracks:\t";
foreach (sort(@atracks)) {
print "$_ ";
}
print "\n";

print "Subtitles:\t";
foreach (sort(@stracks)) {
print "$_ ";
}
print "\n";
Save this file as /usr/bin/mkvinfo.pl and make it executable. Move to the folder holding the archived MKV recordings and use a simple wrapper script:

for i in *.mkv; do
        echo "Checking $i"
        mkvinfo "$i" | /usr/bin/mkvinfo.pl
done
 
As result you will get a list of filenames (your mkv files) and indented a list of audio and subtitles tracks. By visual comparison you can easily spot a missing track and re-record that episode with a Record task in MythTV.

Freitag, 25. Mai 2012

Archive mythtv recordings to mkv using mythnuv2mkv.sh

Even with the biggest hard disks your recordings will use all the available space some time. Yes, mythtv does deletes old recordings - but some of them you want to keep and archive forever.

There is a nice little script called mythnuv2mkv.sh. The original website is now down, the script has not been updated from the author in the last two years (last published version was v1.63). An updated and extended version is available at http://pastebin.com/ENMjrLK (v1.92). The script runs almost out of the box with mythtv 0.25 and below are the required steps.

Download the script and store it as /usr/bin/mythtvnuv2mkv.sh. Open the script in an editor of your choice and modify around line 2277 and locate:
cat > ${OUTBASE}_tags.xml <<-EOF
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE Tags SYSTEM "matroskatags.dtd">
<Tags>
<Tag>
<Targets>
EOF
 and add two lines before this block:

# Remove old file if there
[ -f ${OUTBASE}_tags.xml ] && rm -f ${OUTBASE}_tags.xml 
Save the script, chown owner and group to mythtv and set sticky-group bit.

As the script is quite IO intensive I have added an old 32GB USB stick to my system. The stick becomes available as /mnt/usb-tmp and will hold archived recordings.

Living in Europe I am receiving DVB-S in three formats: 576 (SD format, old resolution), 720p (HD) and 1080p (FullHD). 16:9 and 4:3 format will be automatically identified from mythtv. For each resolution I need one user job in MythTV Backend:

User Job1: Archive 1080: LC_ALL=C /usr/bin/mythnuv2mkv.sh --jobid=%JOBID% --quality=1080 --pass=two --denoise=OFF --deblock=ON --deleterec=OFF --crop=OFF --copydir=/mnt/usb-tmp --chanid=%CHANID% --starttime=%STARTTIME%

User Job2: Archive 720: LC_ALL=C /usr/bin/mythnuv2mkv.sh --jobid=%JOBID% --quality=720 --pass=two --denoise=OFF --deblock=ON --deleterec=OFF --crop=OFF --copydir=/mnt/usb-tmp --chanid=%CHANID% --starttime=%STARTTIME%

User Job3: Archive 576: LC_ALL=C /usr/bin/mythnuv2mkv.sh --jobid=%JOBID% --quality=576 --pass=two --denoise=ON --deblock=ON --deleterec=OFF --crop=OFF --copydir=/mnt/usb-tmp --chanid=%CHANID% --starttime=%STARTTIME%

Make sure /mnt/usb-tmp is writeable for mythtv user and group. The LC_ALL=C definition in front of each command is important for mkvmerge utility which will crash without.

Now to archive a recording you can add a cutlist to a recording and use the menu bar to start one of the three defined archive jobs. Archiving takes time - on my Quad-CPU 90 minutes 720p to mkv need three hours.

Results: Usually the file size are decreased from 10 GB to 2.8 GB.

You can also tweak some defaults in the beginning of the script (ogg vs. faac audio system, mkv or other containers, number of threads etc.).

To avoid stutter while playback you may consider to add a nice -19 to all three jobs and limit the number of concurrent jobs in the backend to two.

Mittwoch, 22. Februar 2012

mythtv and NAS storage for recordings

Since 0.22 mythtv introduced the Storage Group feature: multiple single drives are used to store the recordings or archived movies. The system uses some clever routines to utilize the least utilized (in terms of IOPS or free space) for new recordings and other IO activity.

A loss of a single drive will only impact the recordings on this drive. It's not like a RAID array where a loss of a drive may impact RAID performance or redundancy.

This system is nice and clever for recordings, which can be re-recorded when a show is again in the schedule. More drives allow smooth recordings of multiple recordings at once.

But for archived recordings I prefer an external RAID array - Synology, QNAP - whatever in RAID6 mode. This solution allows to loose two drives out of the array and still have your data available - which does not replace a backup, but protects against the loss of a single drive. Using mythnuv2mkv.sh script I can archive recordings (with an impact on my power bill, as the sytem takes ages to archive HDTV recordings) on the NAS together with my pictures.

But you do not want to wake up the NAS system for every recording on a mixed frontend/backend system. The NAS spindles down when not in use - so I need a solution which actives the NAS system when I start mythfrontend via mythwelcome, but keeps it idle when just recording a show and goes back to sleep.

To do this I modified /usr/share/mythtv/mythfrontend.sh (mythbuntu 11.04) system. As this file is being overwritten with the daily nightly updates, I need a solution to keep my changes to the file.

So each change is being backuped in the same directory and named as mythfrontend.sh.YYYYMMDD (Year, Month, Day without spaces).

The following two lines in /etc/rc.local will replace with each reboot the standard script with the latest backup:

# Replace mythfrontend.sh with our version
# Backup files have .YYYYMMDD as suffix - use latest
cp -f `ls /usr/share/mythtv/mythfrontend.sh.2* | sort -r | head -1` \
      /usr/share/mythtv/mythfrontend.sh
Now I just need to have one backup file with my changes to mount the NAS filer when mythfrontend will be started via lirc and mythwelcome.

The actual start of mythfrontend.real is being initiated in the last if-block in mythfrontend.sh. Use vim +95 mythfrontend.sh to change the code to:
    if [ "$IGNORE_NOT" = "0" ]; then
        /usr/share/mythtv/mythmount.sh
        /usr/bin/mythfrontend.real "$@"
        /usr/share/mythtv/mythunmount.sh
    fi
and backup the modified mythfrontend.sh to mythfrontend.sh.YYYMMDD (replace of course YYYY, MM and DD with the actual date).

mythmount.sh and mythunmount.sh are two new scripts which mount and unmount the NAS shares to /mnt. /mnt has four prepared mount points, FSK00, FSK06, FSK12 and FSK16 (which are the age classifications in Germany, without, six years, twelve years, sixteen years).

Now create /usr/share/mythtv/mythmount.sh:
#!/bin/sh
# Mount NFS shares
for SHARE in FSK00 FSK06 FSK12 FSK16 music pictures FSK00 FSK06; do
    # Check if already mounted (multiple starts of mythfrontend)
    MOUNTED=`mount | grep "/mnt/${SHARE}" | uniq`
    # Mount if not already mounted
    [ "$MOUNTED" ] || mount /mnt/${SHARE}
done
make it executable for everyone and change ownership to mythtv:mythtv. You see that the first two shares are again in the for-list at the end - sometimes the NAS requires too long to spindle up and become available that first two mount actions will time out. Adjust as needed.

Create mythunmount.sh as well:
#!/bin/sh
# umount NFS shares
for SHARE in FSK00 FSK06 FSK12 FSK16 music pictures; do
    # Check if already mounted (multiple starts of mythfrontend)
    MOUNTED=`mount | grep "/mnt/${SHARE}" | uniq`
    # umount if already mounted
    [ "$MOUNTED" ] && umount /mnt/${SHARE}
done
 and change again ownership and executable-bits.

To automatically mount the shares you also need to prepare /etc/fstab and add a line for each share:
//192.168.1.17/movies-fsk0    /mnt/FSK00    cifs    username=mythtv,password=XXX,uid=107,gid=112,iocharset=utf8,user,noauto    0    0
//192.168.1.17/movies-fsk6    /mnt/FSK06    cifs    username=mythtv,password=XXX,uid=107,gid=112,iocharset=utf8,user,noauto    0    0
//192.168.1.17/movies-fsk12    /mnt/FSK12    cifs    username=mythtv,password=XXX,uid=107,gid=112,iocharset=utf8,user,noauto    0    0
//192.168.1.17/movies-fsk16    /mnt/FSK16    cifs    username=mythtv,password=XXX,uid=107,gid=112,iocharset=utf8,user,noauto    0    0
Of course this requires an user account named mythtv with password XXX on the NAS solution. Adjust uid and gid settings to your system and change the IP address to the one from your NAS device.

Now the shares are mounted and unmounted, but mythtv does not see any archived recordings from the NAS, as mythtv searches for files in /var/lib/mythtv/videos, not in /mnt. Just add a symlink for each share into that folder:
lrwxrwxrwx  1 root   mythtv   10 2011-03-31 22:36 FSK00 -> /mnt/FSK00
lrwxrwxrwx  1 root   mythtv   10 2011-03-31 22:36 FSK06 -> /mnt/FSK06
lrwxrwxrwx  1 root   mythtv   10 2011-03-31 22:36 FSK12 -> /mnt/FSK12
lrwxrwxrwx  1 root   mythtv   10 2011-03-31 22:36 FSK16 -> /mnt/FSK16
lrwxrwxrwx  1 root   mythtv   10 2011-03-31 22:36 FSK18 -> /mnt/FSK18
Now after a reboot you should have a working combination of NAS external storage, accessing organized archived recordings while the NAS stays quit when the mixed frontend/backend is recording a new show. Whenever a human being starts mythfrontend via mythwelcome, NAS is being mounted and archived recordings become available.