Up to now, all our cheap Android devices were based on older Linux kernel (3.0.x, 3.4.x) that still used board files (arch/arm/board, but we’ve recently seen companies like Amlogic and Rockchip release source code with Linux kernel 3.10.x. One of the key differences between these version are the move from board files to flattened device tree and multi-platform support. If it is fully implemented, a single kernel image should be able to boot multiple hardware platforms, and all low level configuration handled by the device tree file. Since I’ve connected the serial port of Tronsmart Vega S89 for debugging, and it’s a slow news day, I thought I might try to boot the Linux kernel I compiled myself, but one of the challenge was to get the device tree file. I’ll show how to extract it from the firmware. It should also be possible to get it directly from the flash, but “cat /proc/mtd” does not show a complete list of partition as in previous versions.
I’ve performed the steps below in Ubuntu 14.04. The first thing is to install some tools: the device tree compiler that we’ll use to decompile the dtb (binary) file into a dtd (text) file, and split_bootimg.pl a standard PERL script to extract files from boot.img:
|
1 2 3 4 |
sudo apt-get install device-tree-compiler wget https://gist.githubusercontent.com/jberkel/1087743/raw/5be96af0e1c1346678379b0c0f0330b71df51f25/split_bootimg.pl sudo cp split_bootimg.pl /usr/local/bin sudo chmod +x /usr/local/bin/split_bootimg.pl |
I’ll use M8 / TM8 firmware (Amlogic S802) as an example. The exact procedure will vary between firmware files, but if you can boot.img, the procedure should be platform independent and work for any ARM SoC. After having downloaded and extracted the firmware file (TM8 ap6330_03102014A_0410_ROOT.rar), let’s create a working directory, and unzip the “OTA” file.
|
1 2 3 |
mkdir TM8 cd TM8 unzip ../k200-ota-20140410.zip |
We now get a bunch of files, including boot.img. Great! Time to run split_bootimg.pl script to extract its content:
|
1 2 3 4 5 6 7 8 9 10 |
split_bootimg.pl boot.img Page size: 2048 (0x00000800) Kernel size: 7209567 (0x006e025f) Ramdisk size: 2024995 (0x001ee623) Second size: 17699 (0x00004523) Board name: Command line: Writing boot.img-kernel ... complete. Writing boot.img-ramdisk.gz ... complete. Writing boot.img-second.gz ... complete. |
So we’ve got the kernel, a ramdisk, and a “second file” that happens to be the dtb file. We can now decompile it with dtc (device tree compiler) as follows:
|
1 |
dtc -I dtb boot.img-second.gz -O dts -o meson8_tm8.dtd |
That’s it. Here’s M8 device tree file.
I’ve done the same for Tronsmart Vega S89 (Elite). S89 firmware is usually distributed as an IMG file to be used with AML Flash Burning tool, but I haven’t found a way to extract such file yet. however, I’ve found an “OTA” firmware, to be updated via SD Card, on freaktab, and could extract the device tree file for Tronsmart Vega S89 Elite & Vega S89. Both M8 and S89 Elite DTD files are very similar, but the maximum CPU frequency seems to be higher in M8, and there are other apparently minor differences. Vega S89 DTD file appears to be much different however.
Jean-Luc started CNX Software in 2010 as a part-time endeavor, before quitting his job as a software engineering manager, and starting to write daily news, and reviews full time later in 2011.
Support CNX Software! Donate via cryptocurrencies, become a Patron on Patreon, or purchase goods on Amazon or Aliexpress
Did you finally boot up fully working kernel now?
This looks like fun… let me know how I can help.
@m][sko I’m loading the built boot.img via tftp and running it from there. Before it could boot to command line (in the ramdisk), but no Ethernet. With the right device tree file, I get Ethernet, and access to the command line. HDMI output seems to work but it’s all black. I was expecting the Android UI to show up as well. It’s probably because I need to modify the ramdisk. There are some pinmux conflict in the log as well, because of a camera module which is not present in the stock firmware. Expect progress to be slow, as I… Read more »
hey cnxsoft, don’t know if it’s of any use – but I extracted the partitions with boot.img etc. on my Beelink 16GB (Tronsmart Standard S89 hardware-a-like) if those files are of any use to putter around in?
I’m mostly a windows guy, I need to set up a *nix box for playing around on (mm.. maybe I could use my raspberry pi for that).
M8 is better then S89 Elite?
Antutu benchmark? I Can’t find anything good about m8 (S802).
Thanks
@Dante Both are about the same. But when I tried M8 tended to hang from time to time. It could have been hardware (overheat) or firmware issue. Not sure. @gizmomelb Did you get the boot.img from the NAND Flash or firmware file? If from the NAND flash, how did you do? I can’t find the partitions in my Vega S89 Elite. If you are a Windows guy, and want to play around, you’d better install VirtualBox + Ubuntu on your windows PC, especially if you’re going to give a try at building the kernel. The Raspberry Pi use would be… Read more »
hey cnxsoft – I extracted the boot.img from the NAND using adb shell and some help from Finless Bob on freaktab.
http://www.freaktab.com/showthread.php?12472-factory-default-firmware-images&highlight=
I have the extracted files from the 100k4 NAND, I’ve since updated to 101k4 but haven’t extracted the files again (is there any point if we can unpack the firmware update?).
I’m happy to help out in any way I can. I have two Beelink M8 round boxes, one for testing – my USB -> serial cable/adapter arrived today as I need to update my Gotek floppy emulator (another project).
@Dante
Hi Dante! The square S802 boxes are going by the M8 brand, however Beelink have a round (Tronsmart S89 clone essentially – same motherboard, different firmware but you can flash with the S89 firmware) S802 device which is named M8 (and another one named S82). Confusing!
@Jean-Luc Aufranc (CNXSoft)
Hi cnxsoft – yeah I’ll go virtualbox or if I look around I can probably find an old VMDK VMware ubuntu image I was using to play around with thin client firmware from quite a few years back.
@gizmomelb
Extracting from NAND is useful if you don’t have the firmware, which may happen for some low cost devies in the future with zero support. I got confused when I tried cat /proc/mtd…, but “ls /dev/block/” is what I should have done.
Now I’m at the stage where I try to boot the rootfs (Linaro ALIP) from NFS. The usual bootargs method does not seem to work at all, so I’m trying switch_root, which sort of work until the “Connection Manager” starts, and seems to mess with the nfsroot…
@Jean-Luc Aufranc (CNXSoft)
I don’t have any problem with debian 7.0 armhf rootfs with kernel 3.0 over NFS
But it is maybe that you use initrd https://www.ibm.com/developerworks/library/l-initrd/
As I build most amlogic modules as static as I don’t need any hotplug 🙂
I didn’t use linaro(Ubuntu).
@m][sko
my uboot paramters for AML8726-MX,
http://pastebin.com/V6bhWYqh
@m][sko
Finally even if I can’t mount the NFS share directly with bootargs (I had to edit init with switch_root), it’s still importnt to have a proper bootargs in u-boot. So now I can mount the ALIP rootfs. The display is still black, but at least I get network, and USB mass storage support. I’ll document that tomorrow.
For Rockchip (RK3288) the device tree file is located in resource.img.
It can be extracted using kernel/resource_img script found in the Android SDK here: http://www.cnx-software.com/2014/09/02/rockchip-rk3288-android-4-4-2-sdk-and-schematics-released-for-firefly-board/ or there: http://www.cnx-software.com/2014/09/16/how-to-build-android-4-4-sdk-for-rk3288-tronsmart-orion-r28-beelink-r89/
Syntax from kernel folder:
I have a problem: When run the command “split_bootimg.pl boot.img” i’m getting just kernel and ramdisk. I don’t have the second.gz file. What can i do?
@John
What’s the kernel version? If it’s less than 3.7, then there’s no device tree file.
@Jean-Luc Aufranc (CNXSoft)
Yes, my kernel version is less then 3.7. I have 3.4.5. In this what can i do to make my device tree?
@John
So there’s no device tree. Somebody needs to port the platform you use (Allwinner?) to a more recent kernel.
I am in the same situation….no boot.img-second.gz file. LG Optimus Fx3Q, Android 4.1.2, kernel version unknown (I am running a custom one, but probably less than 3.7).
So all it takes is recompiling the kernel against a newer source? Are there any roadblocks/considerations/gremlins associated with such a task?
@Joel If there’s no new port, you’d basically need to become a kernel developer,so if you have no experience that would be quite a roadblock, and even if you have that could be months of work. If your processor is supported in a more recent version of the kernel, you might just have to write the device tree file by yourself, but that assume all required drivers for your hardware are also part of the kernel. There may also be an issue with binary drivers (GPU. Wi-Fi), and if they don’t match your kernel version, you may be out of… Read more »
@Jean-Luc Aufranc (CNXSoft)
Would merging the kernel source with say, the Optimus F6 (a similar device, which I think has a 3.7+ kernel out there) be any easier? Or is that what you had in mind with my original question?
I’ve compiled Linux Kernels before (the Debian way), so merging the kernel sources, although daunting, seems less of a chore than writing a device tree by hand (especially when I don’t know where to start). Sadly, although similar, the F6 tree does not work for this device (ROM won’t compile).
Thanks for your opinion.
@Joel
If you have a device tree kernel for another device with the same processor, then it will help a lot, but you’ll still need to modify the device tree file yourself, and maybe some other bits too in the source code. It won’t be from scratch so most probably easier. Disclaimer: I’ve never done that myself.
Check and see if the device tree is exposed in /proc. Exposing it is a kernel option. Also the dtc compiler is part of the Linux kernel tree. You don’t have to build it separately. It is possible to append the DTB onto the end of the kernel image, in that case it is not a separate file. It is still there, just harder to get to. Just because these chips are on Linux 3.10 does not force them onto device tree. I have several systems here still using board files on 3.10. I really, really wish these vendors could… Read more »
I got the blog has incorrect magic number error.
dtc -I dtb boot.img-second.gz -O dts -o meson8.dtd
DTC: dtb->dts on file “boot.img-second.gz”
FATAL ERROR: Blob has incorrect magic number
@Li
I’m not sure why.
FTD_MAGIC is defined in http://www.cs.fsu.edu/~baker/devices/lxr/http/source/linux/scripts/dtc/libfdt/fdt.h#L44
and checked in flatree.c @ http://www.cs.fsu.edu/~baker/devices/lxr/http/source/linux/scripts/dtc/flattree.c#L808
Maybe you could try to edit dtc source code to display the magic number you get, and work from there.
cnxsoft : I’ve done the same for Tronsmart Vega S89 (Elite). S89 firmware is usually distributed as an IMG file to be used with AML Flash Burning tool, but I haven’t found a way to extract such file yet A quick follow up on that, it’s only for Amlogic but i found a way to get your dtb out of an .img update (not ota zip) blob. I’m using Amlogic Customization Tool (apparently official tool sourced from some russian forums). It’s purpose is to edit an existing img data file and basically have access to any part of the data,… Read more »
I try to copy text from the examples, and a bar appears over what I’m trying to copy. I select from below, copy, and it copies a bunch of line numbers. I triple-click to select a line, and the whole thing fades into a different style and discards my selection, so I have to make it again. I try to select another box, the bar appears and moves what I just tried to select down. Dude, they’re boxes of text. Stop trying to be more clever than you are with this nonsense. It’s bad enough that you have a wall… Read more »
@ugh
The problem is that it’s not “just text”, and that’s why I have to put it in a pre section.
The plugin I used is called Crayon Syntax Highlighter, you can learn more about it @ https://wordpress.org/plugins/crayon-syntax-highlighter/. It’s being used on over 50,000 websites.
You could learn how to use the icon on top, or any easy way to switch to text few is to double click in the text zones.
New device tree (draft) specifications: https://github.com/devicetree-org/devicetree-specification-released/blob/master/prerelease/devicetree-specification-v0.1-pre1-20160429.pdf
cnxsoft,
I will like to know, I have three different boxes, two uses the same S802 and one S812 and have Openelec.zip for all and the original .img and .zip. The architecture I reckon are all different ie: Ethernet chip etc.
I was able to pull the DTD for the M8 from your instructions above from the original K200-ota file fine, and I am sure can do the same for OpenELEC.zip. my question is.
If I have the LibreELEC.zip how can I put this DTD into it to boot? As I am having bootings issues with LE on the M8.
@Jeff
I’d assume LibreELEC should work the same way as OpenELEC.
cnxsoft,
What I will like you to help me with is how to put this DTD I pulled into the a source code. You showed us how to pull the DTD of any Amlogic FW; now, can you show us how we can put this information into any source code.
@Jeff
It’s been a while, but if I remember correctly, firmware structure are similar between boards, so I just put it back where I found it in the first firmware.
If you want to include it in Linux source tree, I think DTS files are located in arch/arm/boot/dts. Something like that.
i have a mtk device and i cant build kernel and our provider doesnt send an ota update how can i do? Can i try fake ota update? Thanks..
I didn’t get this command
” Writing boot.img-second.gz … complete. ”
I got the kernel and ramdisk file. so no dtb file extracted.
Well. I need these all foe compiling CM13
@Annonymous
What’s the Linux kernel version? It needs to be Linux 3.4 or greater I think, as before board files were used instead of DTS.
I didn’t get this command
” Writing boot.img-second.gz … complete. ”
I got the kernel and ramdisk file. so no dtb file extracted.
Help!
@Vincent
Read comment #36.
@Jean-Luc Aufranc (CNXSoft)
Hi,
Thanks for the quick reply. The device is Xiaomi Mi 5, Kernel version 3.18.20
So what should be done in my case?
Thanks!
@Vincent
In that case, that’s odd. Xiaomi Mi 5 should be quite popular, so maybe ask on XDA forums where the DTS file may be.
Hi, My entire mobile firmware file link is here ==>
https://drive.google.com/file/d/0B1dRSby1cq8jU3FiOG1fZjFsVEk/view?usp=sharing
Would you please give me device and vendor tree??
@Shams Sayied
I can see you have boot.img there, so you can easily extract it if you have a kernel with device tree support.
@Jean-Luc Aufranc (CNXSoft)
Bro there is no kernel for my device.
Why my Second size: 0 (0x00000000)?
Page size: 2048 (0x00000800)
Kernel size: 4759152 (0x00489e70)
Ramdisk size: 1690049 (0x0019c9c1)
Second size: 0 (0x00000000)
Board name:
Command line:
Writing boot.img-kernel … complete.
Writing boot.img-ramdisk.gz … complete.
—-
I used boot.img size 8M from Samsung S3 device
dtb.img file extracted from boot.img this way, not compatible with latest Libreelec, Ubuntu images. Do you know why?
@eugene28
What makes you think the dtb is not compatible?
What is the use of this .dts file?
How to use it to compile kernel without source using a similar device’s kernel source ?
@Tony
The dts file defined the hardware using text. It needs to be converted to binary (dtb), and you can replace the one for in the firmware for another (similar) device with your own, and it may work.
I’ve been searching high and low for a way to make an 800×480 LCD with HDMI input work with my h96 pro TV stick, and I’ve got an idea that the reason might be that the resolution the LCD needs isn’t defined in the DTS/DTB. Does that sound likely? I suspect the stick has been built just to deal with modern resolutions for TVs, of 720p and above (up to 4K with this one), and it has no information about how to output to a display that reports such a low resolution. I’ve gone through all the other things I… Read more »
@orinoco77
For HDMI resolution problems, it’s often an EDID issue. Not sure about it works in Android, but a search for EDID Android will return various links that look potentially useful.
cnxsoft : @eugene28 What makes you think the dtb is not compatible? Message like FATAL ERROR: Blob has incorrect magic number makes him think of compatibility… There is a problem with a binaries format misunderstanding One good man found the reason of magic number inconsistence. (Got from asm code, it looks for 0xD00DFEED and jumps to the error message proc else)) https://ekasiswanto.wordpress.com/2017/08/16/from-boot-img-to-boot-img-dt-to-dtb-and-dts-in-windows-os/#comment-2171 There is at least two formats of the DTB binaries. ‘Simply’ DTB and ‘dtb.img’ – a stack of DTB overlays in one binary. Last ones are extracted from the compiled boot images, then should be splitted in to… Read more »
Quite confused with the dtb files. Many boot image actually does not have dtb files appended to it, however the burn image contains dtb files. I think maybe the dtb files appended to boot image are probably ignored, instead they are picked up at certain location on the flash. On the amlogic device, there is a /dev/dtb. That’s probably where the dtb image is supposed to be stored.
I was able to split a multi-dtb image into several single dtb images using this tool: https://github.com/Wilhansen/aml-dtbtools I’m also very confused with how dtb works with u-boot. There does not seem to be much documentation available. The /dev/dtb does not seem to be a partition but rather provided by the kernel after boot. Here are my findings using an amlogic S912 device and playing with u-boot. 1) Partition information: gxm_q201_v1#mmc dev 1 emmc/sd response timeout, cmd8, status=0x1ff2800 emmc/sd response timeout, cmd55, status=0x1ff2800 init_part() 293: PART_TYPE_AML [mmc_init] mmc init success switch to partitions #0, OK mmc1(part 0) is current device gxm_q201_v1#mmc… Read more »
I am doing this on a samsung phone, and wen i run the dtc -I dtb boot.img-second.gz -O dts -o meson8_tm8.dtd command I don’t get a boot.img-second.gz file. My kernel version is 3.18.24. Can you please help me?