Canon HF200 on Linux/Kubuntu 9.04

Being an 100% Linux user, the software shipped with Canon HF200 is not useful for me because it’s Windows only. Anyway, I’ve tried Pixela Software, the shipped software  on a virtual machine, and I wasn’t convinced…

So far  on Linux there is a complete lack of tools for management the video files that are produced by this camera, namely the MTS file format, despite being able to play them with no problems, only with small quirks… Right now I’m using VLC, and the x.264 codec downloaded from the sources, not the one shipped with Kubuntu and the quality the video is more or less OK. The latest ATI drivers for my ATI 4870 also improved a lot the visualization of the video. Playing the MTS files on the WD TV works 100%, and the quality is superb.

So I can play the files in my computer, BUT for KDE 4.3  Dolphin, it  has no support for thumbnails for MTS (M2TS) files… so I need to open each file to see wich are it’s contents… a painfull task.

It took me a while to solve more or less this issue. It’s not perfect, but it helps:

Using the latest ffmpeg, donwloaded from the sources, I can create JPG thumbnails related to the each MTS file. The idea came from here: http://johannesluderschmidt.de/?p=525

I’ve changed the script so that I can create AVCHD thumbnails:

#!/bin/bash
for FILENAME in *.MTS
do
if test -f “$FILENAME”; then
echo “Create thumbnail of $FILENAME”
ffmpeg -i “$FILENAME” -ss 00:00:01.00 -vcodec mjpeg -vframes 1 -an -f rawvideo -s 180×120 “$FILENAME”.jpg
#cp $FILENAME ~/test1/
else
echo “File $FILENAME not existing.”
fi
done

Save this file with a name like avchdthumb.sh, somewhere on your path, and make it executable with chmod +x avchdthumb.sh

Now you can go to your folder (folders) whre you have stored your MTS files, and just run the command like this for example: /home/primalcortex/tools/avchdthumb.sh

It will take a while, depending of the number of files and the your computer horsepower… I have an I7, so can’t complain…

At the end the result will be for each MTS file a new MTS.jpg file with the initial frame as the thumbnail.

Now I have an idea what’s in the MTS videos.

If thumbnails in video format are needed, like the first 10s, then this URLhttp://code.google.com/p/hdffxvrt/ will do the job. Just change the line for the resolutionof the HF200 to 1920×1080, and execute the script with:  hdffxvrt thumb 0001.MTS. This will create a 10s clip in MOV format named 0001.MTS.mov.

This is a good as it gets. Now I’ll I need is Picasa in Linux to support AVCHD

Regarding the camera itself, when connecting through USB to the Kubuntu, there is no problems in detecting and accessing it through Dolphin, but it’s faster to use a SDHC card reader to copy the files. The camera can create thumbnails for the videos when connecting by USB, but in my case it failed to create ALL thumbnails… weird…

E-Mail problems: a detective story

Few weeks ago, the company where I work for changed the internet provider. Faster access, lower price, you know the drill.

But problems started to appear, namely DNS issues, that took some time to solve, but at the end it worked ok.

Anyway, complains started to appear that people have send e-mails to us, and they were never answered or received…

A quick check on the Postfix logs, showed that those customers where always disconnected:

The smtp transaction was something like:

connect from smtp.domain.com[X.Y.Z.L]
lost connection after DATA (0 bytes) from smtp.domain.com[X.Y.Z.L]
disconnect from smtp.domain.com[X.Y.Z.L]

So It looked like those customer servers connected and disconnected after a few seconds.

A quick search on Google (:-) ) showed that it might be a Firewall issue, a MTU issue or a Postfix bug (!).

We’ve checked, and double checked the firewall. It was ok, and anyway it wouldn’t explain why other server had no problem connecting and transfering mail.

The MTU issue: Ok, we’ve changed the Internet Provider, but all our firewall interfaces are on Ethernet. Anyway we’ve changed the MTU to 1492: ifconfig eth0 mtu 1492. It didn’t solved.Back to square zero.

Postfix bug: It has been working fine on the last 6 months, so very strange… We’ve disable PostGrey (the GLD Daemon). Not a greylisting issue. Bummer.

So we enabled the Postfix debuging features by adding the line: debug_peer_list = smtp.domain.com ( instructions here: http://www.postfix.org/DEBUG_README.html ) and waited for a connection.

Anyway to cut a long story short, we’ve found out that all SMTP transaction was OK until the DATA portion, where it looked that the other e-mail server disconnected. What we did see was that we had a long list o RBL’s to check, and it was taking to long to check every one of them, so it looked like the other e-mail server disconnected after a time-out period even during the email transfer process. I think that is a bug on the other peer, but the issue was ours…

So we’ve cut to half our RBL list, only keeping NJABL, Spamcop and Spamhaus lists, and the servers with problems started to connect and transfer right away the stacked queued messages to us.

Moral of the story: The longer the RBL list the longer the time to process incoming data. Some email servers will just “barf” at these long times.

Right now: Zero problems.

ASUS P6T (i920) and lm_sensors

To make more or less the temperature sensors to work on this motherboard combo Asus P6T and an I7 920 processor, I’ve followed this instructions.

http://guide.ubuntuforums.org/showthread.php?t=1056681

It kind of works… It’s better than nothing.

EDIT: As of [K ]ubuntu 9.10, the shipped lm sensors package detects and works without any issue. So no need to hack it.

Shorewall firewall on UBUNTU LTS 8.04 server doesn’t start on boot

I have in one of my machines a pretty annoying situation related to the fact if the UBUNTU based firewall reboot’s, the firewall doesn’t start automatically…

This is pretty annoying because it means that after a power failure, there is a need for manual intervention to restore exterior access trough the firewall to internal servers.

The solution?

Well I’ve made a bash script, named resetfw.sh that checks the server uptime and if it falls bellow a threshold of 10 minutes, it restarts the firewall:

DAYS=`uptime | cut -d ‘ ‘  -f   4`
HOURS=`uptime | cut -d ‘ ‘  -f   6`
HOUR=`echo $HOURS | cut -d ‘:’ -f   1`
MIN_NP=`echo $HOURS | cut -d ‘:’ -f   2`
MIN=`echo $MIN_NP | cut -c 1-2`

if [ $DAYS = "0" ]; then

if [ $HOUR = "0" ]; then

if [ $MIN -lt "10" ]; then

/etc/init.d/shorewall stop
/etc/init.d/shorewall start
/etc/init.d/shorewall stop
/etc/init.d/shorewall restart

logger “Firewall reset due to reboot: Uptime on action: $DAYS days, $HOUR:$MIN”
fi
fi

fi

Then all we have to due is to run this script periodically through the crontab:

*/5 * * * * /root/resetfw.sh

InstallShield execute error

I’ve hit this error when running an InstallShield setup program on Linux RedHat AS 4.0:

ServiceException: (error code = 305; message = “ISDatabase is unavailable: b22e544779b311a7661cb6eff5bf86d8/isdb”; severity = 0)
at com.installshield.wizard.service.AbstractWizardServices.getISDatabase(Unknown Source)
at com.installshield.util.LocalizedStringResolver.initialize(Unknown Source)
at com.installshield.wizard.service.AbstractWizardServices.addStringResolverMethod(Unknown Source)
at com.installshield.wizard.service.LocalWizardServices.addStringResolverMethod(Unknown Source)
at com.installshield.wizard.service.AbstractWizardServices.createStringResolverMethods(Unknown Source)
at com.installshield.wizard.service.AbstractWizardServices.<init>(Unknown Source)
at com.installshield.wizard.service.LocalWizardServices.<init>(Unknown Source)
at com.installshield.wizard.service.WizardServicesFactory.createLocalWizardServices(Unknown Source)
at com.installshield.wizard.Wizard.getServices(Unknown Source)
at com.installshield.wizard.Wizard.setWizardValue(Unknown Source)
at com.installshield.wizard.Wizard.initializeWizardPropertyValues(Unknown Source)
at com.installshield.wizard.Wizard.startup(Unknown Source)
at com.installshield.boot.BootMain.boot(Unknown Source)
at run.main(Unknown Source)
ISDatabase is unavailable: b22e544779b311a7661cb6eff5bf86d8/isdbWARNING: could not delete temporary file /tmp/ismp001/9681561

The error was related to the language locale:

[myservermachine opt]# locale
LANG=en_US.UTF-8
LC_CTYPE=”en_US.UTF-8″
LC_NUMERIC=”en_US.UTF-8″
LC_TIME=”en_US.UTF-8″
LC_COLLATE=”en_US.UTF-8″
LC_MONETARY=”en_US.UTF-8″
LC_MESSAGES=”en_US.UTF-8″
LC_PAPER=”en_US.UTF-8″
LC_NAME=”en_US.UTF-8″
LC_ADDRESS=”en_US.UTF-8″
LC_TELEPHONE=”en_US.UTF-8″
LC_MEASUREMENT=”en_US.UTF-8″
LC_IDENTIFICATION=”en_US.UTF-8″
LC_ALL=

As you can see the language is set with UTF-8. This is the root cause for the problem. Just do the following:

export LANG=”en_US”

and try again. At least it works for me.

FileMon for Linux (sort of…)

In Windows, Sysinternals (now Microsoft) allows to check if a process access a file and if it has trouble accessing it like lack of permissions, and even if something is missing….

For Linux, an alternative is to execute the strace program:

strace -fo outputfile command&

tail -f outputfile

In real time the process activity can be seen. Because the output is in the outputfile you can after the process termination process it with tools like grep, cut and so on.

The only thing missing is a GUI for the commands…

Ubuntu/Kubuntu, Linux and Mono

Because of several issues regarding licensing, patent traps, Microsoft, Silverlight, and so on, I’ve completely remove mono from my Kubuntu desktop, and all associated applications (that where none as far as I can tell).

So there it is. You can live without .NET on free platforms.

PS: By the way, I’ve notice something very, very funny: Miguel de Icaza blog is: tirania.org. (Never read it or be there…)

Well in Portuguese tirania is a word that defines an opressive regime, dictatorship… Coincidence?

Moving from windows to Kubuntu on a business work computer – Part I

I’m moving my work laptop to Kubuntu leaving Windows behind. Despite my company having Microsoft licensing for all computers and for a lot of products, I’m jumping out of the bandwagon…

The computer is an HP nx9420 with and 100G SATA hard disk, and so far so good, everything looks to work fine, including Wireless, Bluetooth, except the integrated smartcard reader. But this can be a problem on the OpenCT software, not on Kubuntu Hard Heron 8.04 release. Using the Gemplus USB reader works perfectly.

Evolution for connecting to exchange does work, but it has some stability issues and some quirks that I find annoying… but otherwise it can replace Outlook without any problem, and have at the same time email accounts open like Gmail, and other IMAP mail accounts. An annoying thing is it can’t remember already used e-mail addresses, or looking at the Personal directory if your using the Global catalog server for finding emails so you have to type them if they are not on your GC. I must look for a solution to this.

VMWare server and VMware converter allows to run some software that I really need and has no counter part on Linux World, like Visio and the proprietary software that my company works with.

Also 2X Application server allows you to run Windows applications directly on your Linux Desktop, so you don’t even need to use the VMWare console to work!. And it’s free for 5 connections after the 30 day trial period.

I’ll get back later on this

Ubuntu: The perfect mail gateway server

I’m setting up a new firewall, email gateway server using Ubuntu 8.04LTS. As in a previous post, I’m following (not always by the book) the Howtoforge to setting up a email/spam gateway Ubuntu mail gateway.

Three things didn’t go quite as expected, and so here they are engraved for ever in the internet stone:

1st) General: In the Webmin Others->Server and System Status, the Apache server monitoring reports apache as always down. The solution is to go to Servers->Apache Webserver and select on the top the link Module Config. At the bottom the option Path to Apache PID file is set to Automatic. Change it to point to /var/run/apache2.pid and save. The webmin monitor now for apache should show the apache status as OK.

2nd) Mail: I really like Mailscanner and it’s partner Mailwatch. One of the issues that I was having was that the messages that where in quarantine, Mailwatch didn’t show any option to delete or release the messages for them. The issue was lack of folder permissions. Mailwatch couldn’t access the quarantine directory. So executing the command chown -R postfix:www-data /var/spool/MailScanner and chown -R postfix:www-data /var/lib/MailScanner/ did the trick, and I can now delete and release quarantine messages.

3rd) Mail relay: After setting up postfix, all incoming messages where refused on the external interface because postfix denied relaying. Please note that I’m using this server to receive mail from the internet, to see if a mail is safe, no virus, no spam, no phishing, and so on, and then forward it to our internal mail servers. So I have a relay_domains file that describes our domains, and a transport file that specifies where are the “real” mail servers, but even with this, postfix was always refusing the mails.

The solution for this issue is on the empty mydestinations option. Setting up this option with mydestination = hash:/etc/postfix/relay_domains and stoping and restarting postfix, did the trick.

Regarding the howtoforge manual, I skiped some of the things, like fuzzyocr, and removed Bind9 from the server.

As I progress in setting up and configuring the the server, if anything worth mention, I’ll post it here.

Linux mail gateway

I’ve run where I work for 4 years a Mandrake based firewall with Postfix and Mailscanner. I really, really liked mailscanner, but for my colleagues the setup was “too complicated”. So I moved to EFW, Endian Firewall comunity edition. What it brings in ease of use it lacks in flexibility.

Finally my prayers where listen, and I’m going to move again to a custom build full fledged mail gateway with Mailscanner. Check out: this howto.