Fix magic for MP4 MSNV in Ubuntu 12.04

In Ubuntu, as in other flavours of Linux, utilities like “file” detect the type of files by using an algorithm based on “magic numbers”. This algorithm looks for fixed fields in the examined file. Lately, some of these utilities have failed in detecting the proper type for some encountered MP-4 files which do not contain the “magic numbers” expected of a MP-4 file.

This note presents a method to update the magic files to detect new file formats. More specifically, the method concentrates on updating the magic files to recognize the new MP-4 format.

This note relates to:

  • Ubuntu 10.04
  • Ubuntu 12.04
  • MP-4 format, major brand MSNV

References:

This note uses a number of examples to illustrate the fix. In these examples, two files are used, a MP-4 file recognized by the file utility named “known.mp4”; and, a MP-4 file not recognized by the utility named “unrecognized.mp4”.

Exploring the problem

For the recognized file, the following sequence is experienced:

As seen above, the format of the file is recognized correctly and the mime-type is return appropriately. Here is the sequence for the file which is not recognized correctly.

In the case of the file which is not recognized, a generic mime-type is returned.

Differences between files

Programs from the FFMPEG or libav projects can be used to find out details of video files. Using avprobe (or ffprobe), it is possible to get details from each file and examined the differences.

By looking at these reports, it becomes evident that although both files are reported as MP-4, they are not reporting the same major brand.

Detailed Differences

The “od” utility can be used to look at the byte information from each files. This information is necessary to craft the magic file needed to recognized the new format.

A different file header is observed from the file that is not recognized:

In the case of the recognized file, the bytes detected are from offset 4 to 11, where “ftypisom” is recognized as belonging to a file of type MP-4. The same bytes in the file not recognized are “ftypMSNV”. The crafting of the magic numbers to recognize this new file type must be based on this latter string.

Updating Magic Numbers

This note does not explain the format of the magic number files. This topic is covered in other pages. See the references above.

In Ubuntu, updating the file /etc/magic with the proper entries is all that is required for the file utility to detect the new types. There is no need to compile the file, as in other versions of Linux. Nor is there a need to perform any reboot.

To edit the magic number file:

The following lines should be added to the magic file:

After the magic number file is update, save and close.

Testing

To test that your changes are working, perform a “file” command on the previously not recognized video file:

If the results above are not observed, then something has gone wrong.

Conclusion

The same problem was encountered on Ubuntu 10.04 and the fix introduced above corrected the problem. I suspect that all intervening versions of Ubuntu between 10.04 and 12.04 suffer the same problem and that the same fix applies.

Leave a Reply

Your email address will not be published. Required fields are marked *