2012
08.13

Fix magic for MP4 MSNV in Ubuntu 12.04

In Ubuntu, as in other flavours of Linux, utilities like “file” detect the type of files by using an algorithm based on “magic numbers”. This algorithm looks for fixed fields in the examined file. Lately, some of these utilities have failed in detecting the proper type for some encountered MP-4 files which do not contain the “magic numbers” expected of a MP-4 file.

This note presents a method to update the magic files to detect new file formats. More specifically, the method concentrates on updating the magic files to recognize the new MP-4 format.

This note relates to:

  • Ubuntu 10.04
  • Ubuntu 12.04
  • MP-4 format, major brand MSNV

References:

This note uses a number of examples to illustrate the fix. In these examples, two files are used, a MP-4 file recognized by the file utility named “known.mp4”; and, a MP-4 file not recognized by the utility named “unrecognized.mp4”.

Exploring the problem

For the recognized file, the following sequence is experienced:

As seen above, the format of the file is recognized correctly and the mime-type is return appropriately. Here is the sequence for the file which is not recognized correctly.

In the case of the file which is not recognized, a generic mime-type is returned.

Differences between files

Programs from the FFMPEG or libav projects can be used to find out details of video files. Using avprobe (or ffprobe), it is possible to get details from each file and examined the differences.

By looking at these reports, it becomes evident that although both files are reported as MP-4, they are not reporting the same major brand.

Detailed Differences

The “od” utility can be used to look at the byte information from each files. This information is necessary to craft the magic file needed to recognized the new format.

A different file header is observed from the file that is not recognized:

In the case of the recognized file, the bytes detected are from offset 4 to 11, where “ftypisom” is recognized as belonging to a file of type MP-4. The same bytes in the file not recognized are “ftypMSNV”. The crafting of the magic numbers to recognize this new file type must be based on this latter string.

Updating Magic Numbers

This note does not explain the format of the magic number files. This topic is covered in other pages. See the references above.

In Ubuntu, updating the file /etc/magic with the proper entries is all that is required for the file utility to detect the new types. There is no need to compile the file, as in other versions of Linux. Nor is there a need to perform any reboot.

To edit the magic number file:

The following lines should be added to the magic file:

After the magic number file is update, save and close.

Testing

To test that your changes are working, perform a “file” command on the previously not recognized video file:

If the results above are not observed, then something has gone wrong.

Conclusion

The same problem was encountered on Ubuntu 10.04 and the fix introduced above corrected the problem. I suspect that all intervening versions of Ubuntu between 10.04 and 12.04 suffer the same problem and that the same fix applies.

2012
08.11

After upgrading to Ubuntu 12.04 (Precise Pangolin) on a Dell Inspiron 1720, the speakers continued playing even after plugging in a set of headphones. This problem has been reported and fixed in multiple forums. However, the fix requires a model number, which can not be easily guessed. Therefore, this post is meant to help those who have a machine similar to the Dell Inspiron 1720.

References:

This note applies to:

  • Ubuntu 12.04
  • Dell Inspiron 1720

The fix is applied by using the following steps:

  • Edit /etc/modprobe.d/alsa-base.conf and modify the appropriate line
  • Reboot computer

Edit alsa-base.conf

Initially, the file contained the following line:

This line must be commented out (in case you want it back) and replaced as follows:

After saving the file, rebooting the computer is all that is required.

2012
06.25

This note explains how to add new public IP addresses, in excess of the first static IP address, to the WAN interface of a DD-WRT router. All public addresses are assigned to computers that reside within the LAN network served by the router. Therefore, network address translation (NAT) is performed between the public addresses and the addresses assigned internally. This notes is written after it was successfully performed on:

  • ASUS RT-N16 Wireless Router
  • DD-WRT v24-sp2 (08/07/10) mega (SVN revision 14896)
  • ISP: TekSavvy

References:

When enquiring to receive a new static IP address from my ISP, I found out that a subnet could be leased for a monthly fee. I selected a subnet containing two IP addresses which was assigned to my internet service. The new addresses were communicated using a slash notation: XXX.XXX.XXX.XXX/30.

Although a /30 subnet might suggest that four distinct addresses are available, there are only two. In fact, each subnet contains two addresses that have special meaning: the first, which is the subnet identifier; and, the last, which is the subnet broadcast address. Therefore, the first address available for assignment in a subnet is the one after the subnet identifier. For example, if the assigned subnet was 1.2.3.4/30, then the subnet identifier would be 1.2.3.4, the broadcast address would be 1.2.3.7 and the available addresses would be 1.2.3.5 and 1.2.3.6.

For simplicity in the following examples, we will assume that the assigned subnet is 1.2.3.4/30. Also, it will be assumed that the public address 1.2.3.5 is assigned to a computer that is manually set with the LAN address 192.168.1.40.

Solution

The solution is two-fold. First, assign one of the new public IP addresses to the router’s WAN interface. Then, use firewall rules to route packets to and from the desired computer within the LAN.

Although the solution calls for entering commands through the router’s web interface, I suggest you test those commands by entering them at the command line using a telnet or SSH session to your router. Once this works well, transcribing the commands to the web interface is a good way to save those changes in case the router is rebooted.

Assign Public Address

Using a web browser, open the web interface to your router. This is usually done by directing your browser to an address similar to: 192.168.1.1.

Direct your browser by selecting the “Administration” tab, followed by the “Commands” sub-tab.

In the text box titled “Commands” under “Command Shell”, enter the commands to assign the public address to the WAN interface. Use the example below as a template and substitute the addresses according to your situation:

Once this is entered in the text box, save the changes by pressing the button titled “Save Startup”.

Firewall Rules

To assign the firewall rules, the text box mentioned in the previous step is used. However, when saving the content of the firewall rules, the button titled “Save Firewall” is used instead.

In the firewall rules, one command is used to map the public address to the internal address; one command is used to map the internal address to the public address, and; one command is used to accept each port that should be forwarded.

Use the following template and substitute the appropriate addresses and ports:

The above example forwards HTTP (port 80) and SSH (port 22) requests to the internal computer.

Reboot Reouter

For the changes to take effect, the router must be rebooted. Using the router’s web interface, navigate to “Administration” tab and the “Management” sub-tab. Finally, press the button titled “Reboot Router” at the bottom of the page.

2012
03.24

Tethering iPhone 5.1 on Ubuntu 11.10

This blog site already features an note on tethering the iPhone to Ubuntu (http://www.bitsbythepound.com/tethering-iphone-on-ubuntu-11-04-397.html). However, since upgrading to iOS 5.1, when the iPhone is connected to the Ubuntu platform, an error is reported stating that “Unhandled Error Lockdown”.

This note applies to:

  • Ubuntu 11.10 (Oneiric)
  • iPhone 5.1

References:

The solution offered by this note is to:

  • Install the package repository offered by Paul McEnery
  • Adjust the package repository to be picked up in oneiric
  • Install the necessary packages from the repository
  • Configure

Install Package Repository from Paul McEnery

Adjust Package Repository for use in Oneiric

I applaud the work done by Paul McEnery in providing us the tools to use the iPhone. Unfortunately, the package repository is not available for Ubuntu 11.10. On the other hand, the package repository for Natty (11.04) seems to work just fine for Oneiric. In this step, the package list is modified to point to the Natty version.

While editing the file, replace “oneiric” to “natty” and save again. The resulting file should look something like:

Install Necessary Packages

Configure

The first step of configuration is to tether the iPhone to the platform using the USB connector. Then, run the following command:

Finally, unplug and re-connect the iPhone. The error should not be reported. Instead, all services from the iPhone should be available.

2012
03.24

Since upgrading to Ubuntu 11.10, some games that used to run perfectly using PlayOnLinux started to display unwanted behaviour. In particular, the mouse jumps from the expected position in Starcraft: Broodwar. This behaviour prevents efficient playing. Researching the Internet shows that many other games are plagued by the same problem.

This note applies to:

  • Ubuntu 11.10
  • PlayOnLinux 4.0.16
  • Wine 1.3.18
References:

Many solutions are offered on the Internet which did not work on my system. However, the problem appears fixed in a newer version of Wine. The approach offered in this note is to install Wine 1.4. The latest version of Wine is not yet available via the packages offered by Ubuntu. However, the latest version (beta) is available via the WineHQ’s PPA. The following steps adds a new package repository and updates Wine to the latest version available from this repository. As an interesting note, Wine 1.4 is available via the package named wine-1.3.

After those changes, you can verify the version of the currently installed version of Wine with the following command.

Conclusion

Wine and PlayOnLinux are great tools. The complexity of the task performed by those tools is such that one must expected some unstable versions. This note demonstrated how a user can jump into experimental versions of Wine to access solutions earlier than pushed by the standard packages.

2011
07.06

I have been using a Solid State Drive (SDD) for some time and I have decided to share my notes on how to do it and references I have followed. The configuration I use is to have two disks attached to my platform: the first is the SSD and holds the Operating System; the second is a magnetic drive which holds my home partition.

This note applies to:

  • Ubuntu 11.04
  • ATA INTEL SSDSA2M080G2GC
  • ext4

From research, the important aspects that need to be dealt with when using a SSD are:

  1. Enable TRIM
  2. Disable access timestamp on files
  3. Adjust disk scheduler
  4. Moving log files to RAM drive

Each of these topics are discussed separately below.

Enable TRIM

TRIM is the process under which an operating system tells a drive that a sector is no longer in use. This is not necessary for magnetic drives and, historically, operating systems did not provide this information to the drives as it increased the amount of information traversing between a host and its drives. However, this information is crucial for SSDs as it is needed to perform proper wear leveling.

Information on TRIM can be found via these links:

In Ubuntu, TRIM is enabled via the file “fstab” located in “/etc”. One needs to find the drive associated with the SSD and add the option “discard”. To edit the “fstab”:

Then add the “discard” option to the drive concerned. In my example, the root drive is the SSD. Therefore, the resulting file looks like:

After the “fstab” file is modified, one must reboot the system for the change to take effect.

After reboot, one should ascertain that TRIM was in fact enabled. To do so, a trick provided by Nicolay Doytchev is used (link above). However, it is claimed that this trick works with ext4. One might not get expected results from other disk format.

1. Become root and move to a directory managed by the SSD:

2. Create a file with random bytes in it:

3. Get and record the disk sector address for the beginning of the file:

Note the number under begin_LBA and use it for <ADDRESS> below:

4. Access file data from disk (replace /dev/sdX for the drive you are enabling in fstab):

The result of this should be random bytes and look like this:

5. Delete the file and synchronize the file system so that changes are pushed to the SSD:

6. Finally, repeat the command from step 4. Reading the same disk sector should yield an empty sector. For example:

Disable access time stamps on files and directories

Every time a file is accessed, a time stamp for last access is recorded with the file. It is believed that this information is generally unnecessary and that it is the source of a lot of wear on a SSD. Turning off access time stamps does not disable last modified time stamps, which are crucial to a lot of computer tools.

However, it is possible that some tools you are using rely on last access time stamps. Therefore, I suggest that you disable access time stamps separately than all other changes you make to your system. Turn it off and then leave your system unchanged for a while to see how it performs under this mode. In that period, try out all your tools. Tools that rely on file caching might not do as well as others.

Interesting links about last access time:

Disabling last access time is done by adding the options “noatime” and “nodiratime” for the SSD to the file “fstab” under the directory “/etc”. First, edit the “fstab” file:

Then, locate your drive and add the options “noatime” and “nodiratime” at the appropriate line. For example, my file looks like:

After the file is saved, one must reboot the operating system for the changes to take effect.

After reboot, you can verify that the recording of access time is not longer in effect with the following tests:

1. Move to a directory managed by the SSD and writable to your user. If your home directory is on the SSD, then the following would work:

2. Create a file for testing purposes:

3. Record the last access time:

4. Wait for a minute or two to elapse (you can use ‘date’) and then read the file:

5. Repeat command from step 3 and compare the results. If the same time is returned, then “noatime” is in effect. If a newer time is returned the second time, then the operating system is still recording access time for the file.

6. Clean up after yourself and delete the test file:

Adjust Disk Scheduler

By default, the CFQ scheduler is used to access a drive. This scheduler is designed for magnetic drives and take into account variables such as seek times. In SSDs, access time is fairly constant. Therefore, there is no need for a complex scheduler and addressing requests on a first-come-first-served basis is adequate. I have used the “noop” scheduler and this is what I demonstrate. However, there is an interesting post at this blog that might convince otherwise.

The trick offered here changes the scheduler after the boot is performed and as the operating system is starting it services. Therefore, the boot process does not benefit of these changes. The post above provides links to enable scheduler at the boot loader.

Myself, I like keeping my boot loader as clean as possible, so I have adopted this trick.

1. Edit the file “/etc/rc.local”:

2. Add the following line by taking care to replace sdX for the proper drive:

This change will take effect at the next reboot. However, to save a reboot, one can apply the change right away with (again, replacing sdX for proper drive):

Moving log files to RAM

Log files are a source of constant writing to disk. Personally, I have not yet made the change to my drive for two reasons:

  1. I am ambivalent about throwing away the logs since I sometimes rely on them to find causes of crashes. Given I keep up with the latest version of Ubuntu and that I have a set of problematic video drivers, I feel I must keep my logs around.
  2. Of all the solutions offered, I have not been able to make one work to my liking. I find a lot of solutions presented on the web are finicky and prone to errors.

However, my research indicates that this will yield to an earlier retirement of my drive. Therefore, I offer here a link that I found useful on this topic:

Other readings

Here are a couple of interesting links that might guide you in your decisions. I do not personally endorse the views presented there. However, I believe they provide good food for thoughts:

Conclusion

With a SSD drive, the time to boot up has greatly decreased and performance associated with my development tools as increased. Surprisingly, the general temperature reported by my laptop sensors has also gone down. However, as most activities performed on computers are “web based”, the network-bound activities have remained pretty much the same.

All in all, I have been quite satisfied with my purchase of a SSD drive.

2011
06.09

Tethering iPhone on Ubuntu 11.04

Reference:

  • Ubuntu 11.04
  • iPhone 3GS with iOS 4.3

The following steps can be used to enable tethering between a platform running Ubuntu 11.04 and an iPhone. At a high level:

  1. Enable tethering on iPhone
  2. Install repository from Paul McEnery
  3. Install necessary packages
  4. Connect

Enable Tethering on iPhone

The use of iPhone tethering might be governed by your wireless plan. If data tethering is allowed, one should be able to turn it on using the “Settings” application.
In the “Settings” application, choose “Personal Hotspot” from the menu, and enable it by pushing the switch to “on”.

If one can not accomplish this step, the remainder steps from this article will be in vain.

Install repository from Paul McEnery

Adding package “python-software-properties” provides the command “add-apt-repository”. This command is then used to make it really easy to add a repository from PPA.

Install necessary packages

Once the repository is installed, installing the packages is straight forward:

During the configuration of those packages, a kernel module must be built. In the case that the kernel headers are not installed, the following error message is printed on the screen:

Module build for the currently running kernel was skipped since the kernel source for this kernel does not seem to be installed.

In this case, the proper headers must be installed. To find out which headers are required:

Then, install the headers using the following command. Ensure that you replace the proper version according to what was returned previously.

When this command completes, the iPhone module should be built. If not, then the system can be prompted to rebuild the kernel module:

Connect

Connect the iPhone to the platform using a USB cable. When the USB device is detected, an Ethernet should be established automatically.

2011
06.02

Login package updates messages are incorrect

We have a couple of machines running 10.04 LTS and they have been showing messages similar to the following when you ssh to them.

11 packages can be updated.
7 updates are security updates.

And yet, when you do an update, there is nothing to update. There was a bug that caused the package update information to be incorrectly dumped into /etc/motd.tail and never updated. The bug was fixed, but the incorrect /etc/motd.tail was never cleaned up. This can be fixed by removing /etc/motd.tail. The correct information will be generated when available in /var/run/motd.

Reference: http://old.nabble.com/Where-does-the-%22n-packages-can-be-upgraded%22-message-come-from-td31632514.html

2011
04.25

Materialize Maven Modules in Eclipse

This note relates to:

  • Eclipse SDK 3.6.2
  • m2eclipse plugin 0.12.1
  • subclipse 1.6.17

In Eclipse, modules in a Maven project are materialized separately when the project is first checked out of a Subversion repository. However, when new modules are loaded as a result of a repository update, the new modules are not automatically materialized.

To prompt Eclipse to create new projects for the acquired modules:

  1. Right-click on the project containing the new modules
  2. Select Import… from menu
  3. Select Maven > Existing Maven Projects and press Next button
  4. Set the check boxes next to the projects to materialize and press Finish button
2011
04.25

This note relates to Eclipse SDK 3.6.2.

When editing a Java source file, in Eclipse, warnings are reported when an import statement is specified but not used. Unused import statements can be removed automatically by using the Java editor menu entry Source > Organize Imports.

This can be automatized when the file is saved using the following approach:

  1. Open “Preferences” page via menu Window > Preferences
  2. Navigate to Save Actions for Java Editor (Java > Editor > Save Actions)
  3. Select Perform the selected actions on save
  4. Select Organize imports
  5. Press button Apply

From this point on, when a Java file is saved, import statements will automatically be cleaned.