Embedded Linux Development

This page contains information and links for embedded Linux and Android development, and focuses on the “system” aspect (low level, OS configuration), so you won’t find that much information about application development, and many concepts discussed in this page can also be applied to “big Linux” development. Although the information provided is supposed to be target independent, some materials are board specific if it can help understanding more general ideas.

Setting up an Embedded Linux Development Machine – Top

Embedded Linux development is usually done in a Linux machine, so you’ll have to set it up to exchange files between the build machine and the target boards, debug the target board, install a toolchain to cross-compile software, etc…. If you are new to Linux, Ubuntu is a good distribution for ARM Linux development. To get started with Ubuntu 12.04, you may want to download “Getting Started with Ubuntu 12.04“, a free 143-page guide.

Installing a cross-toolchain

Once you have installed Linux in your machine, the first thing you need to do is to install a cross toolchain which is often provided by the Silicon manufacturer, but there are now toochains part of common Linux Distribution such as:

The Yocto Project and Buildroot are tools to generate a full Linux distribution for embedded systems, and they will also generate a cross-toolchain for you. I shortly address those in ARM Linux development rootfs section below.

Installing a server to share files between the build machine and the target board

So now you’ve got a toolchain installed on your machine, you need to setup a method to exchange files between the target board and your machine via the network in order to speed up development. There are several server available to achieve this feat:

  • NFS – NFS stands for Network File System, and in my experience, is the most commonly use to exchange files between the build machine and the target board. After you build your app, you copy it in the NFS server, and run it from there in the target board. Nice and easy. See how you can install NFS in Ubuntu.
  • TFTP – The Transfer File Transport Protocol is usually used to load files in the bootloader, it can be used to load a system image (e.g. ramdisk) via the network either to debug the system (kernel + rootfs) or to perform a network boot in order to save on BOM cost (e.g. no Flash needed) and maintenance, as files stored in a central server may be easier to maintain that accessing individual files. In Ubuntu, you can use apt-get install atftpd or apt-get install tftpd to install a TFTP server.
  • CIFS/SAMBA – Aka “Windows Share”, this is not really used unless you also need to use Windows. Here’s how to configure a samba server in Linux.

You’ll also need to install some other tools such as a version control client (git, svn, cvs…), debuggers, and more. But more on that below. You’ll most probably use a serial console to access the board using minicom or screen (Linux), or hyperterminal or teraterm (Windows).

Configuration Management: Version Control, Bug Tracking and Continuous Integration – Top

This section is not specific to embedded software, but you’ll need a version control system to manage your code, especially if you work as a team, but it’s also useful even if you work by yourself on a project since you can easily keep track of changes. There are several version control systems available, CVS is the oldest, and less and less used, SVN is more recent and still used in some projects, but the most used solution is now Git.

You can install Git yourself on Linux or Windows, or use online services such as Gitorious or Github. Online services are free for public repository and small teams, but you need to pay to host closed source projects.

If you’re still using CVS, I wrote some posts explaining how to generate a ChangeLog automatically, and how to block commit by “naughty” developers that do not write commit comments.

For bug tracking, you can install tools such as Bugzilla or Trac. Bugzilla is the most widely used, and Trac is actually more than just a Bug tracker, as it is used to manage complete projects and integrates with svn. There is a more complete list on Wikipedia.

Finally, you’ll most probably want to automatically build your code regularly, either daily/nightly using scripts for example, either using continuous integration (build after each commit) tools to make sure your build complete. Such builds can also be used by the QA team, as they can indicate which build failed to pass some tests. Some interesting tools include Jenkins-CI, Buildbot or BuildHive (Jenkins-CI on Github).

Developing with Virtual Machines – Top

If you have to use Window XP/7 has your main machine, you can still do Linux development with x86 virtualization packages such as Oracle VirtualBox or VMWare Player, although if you are a team of developers, you could use a Linux server and do development via an SSH terminal window using Putty for example.

Installing Linux distributions in VirtualBox or VMWare Player is really easy, so I won’t explain how to do this here. One of the things you probably want to have is the ability to copy/paste text between the Windows host and the Linux guest OS. I’ve explained how to do this in VirtualBox in a previous post, since this is not enabled by default. The simplest solution to exchange files between the host and the guest OSes is probably to setup a network share.

Emulators and Virtual Hardware Platforms – Top

Whether you are a student with a very limited budget, or a more experienced engineer needing to work on a project before the hardware is ready, you can get started with cross-development using emulators such as qemu, which can emulate a lot of targets such as ARM, MIPS, PCC and more.

Qemu is often provided as a binary package in Linux distributions, and you could get install it using “apt-get install qemu-system” in Debian or Ubuntu for example. In some cases, you may want to use the latest qemu version, in that case you’ll have to build it. Here’s how to build qemu for ARM emulator.

Now that you’ve got qemu up-and-running, you can try it. I’ve written the instructions to emulate a Gumstix Overo board (TI OMAP3 – ARM Cortex A8) as well as ARM Versatile Express in qemu, and you could use this to get started with ARM development.

Transferring files between the host and qemu is not really straightforward, so I’ve explained how to to do either by directly copying files to the image or via the network (NFS/SSH).

There are limits to developing with emulators however due to the low performance and the lack of peripheral support, so usually, anything that accesses the hardware and is graphics intensive (although OpenGL ES is now supported) cannot usually be run in qemu. The solution to this issue is the implementation of virtual hardware platforms that can run on power machines and emulate the CPU(s) and all peripherals of an SoC. I haven’t tried it myself yet, but If you want to have a go, you may want to try Impera OPVsim simulator. It’s free and there are a a few models available.

ARM has recently released fast-models for ARMv8. I’ve written instructions to use it with OpenEmbedded 64-bit ARM port.

ARM Linux development – Top

This section is specific to ARM and Linux, and I assume you’ve already setup your build machine as specified above.

You’ve usually got 3 components in an Embedded Linux systems, which are called in sequence at boot time.

  • The bootloader (e.g. U-boot)
  • The Linux kernel
  • The root file system (rootfs)

I’ll give examples showing how to each, as the method is more as less the same:

Once you get the bootloader and Linux kernel ready, you’ll need a rootfs that contains all userspace files required to complete the system boot. Here are some methods to get a root file system for ARM:

If you want to cross-compile ARM packages you can now use MultiArch in Ubuntu 12.04, This will download and install ARM packages (dependencies), and cross-build packages from sources for ARM targets.

Linux Kernel Development – Top

If you want to learn more about kernel development, there are some must read books:

  • Linux Device Drivers, Third Edition – Available for free @ https://lwn.net/Kernel/LDD3/ for Linux 2.6 drivers and beyond. In case you prefer an hardcopy it’s also available on Amazon.
  • Linux Kernel Development (3rd Edition) – Available on Amazon as paperback or kindle editions.
  • Linux Kernel in a Nutshell – Available for free online.

There’s also a great list of books and how-tos about Linux kernel development on Linuxquestions.org

Once you’ve developed drivers for the kernel and/or fix a bug, you’ll want to submit a patch, but you’ll have to follow the rules in the provided link or it will be declined.

Embedded Android Development – Top

Whether you plan to develop apps for Android, or just want to tinker with the system (with ADB), you’ll need to install the Android SDK. Here’s how to install the Android SDK on Windows XP and Ubuntu 12.04

Google provides Android tutorial for application development. I know it’s not related to to embedded Android, but still interesting. You can read the summary, or go directly to the tutorials.

If you want to learn more about Linaro Android and how to install it on a Pandaboard, you can do so with the Linaro Android Tutorial by Zach Pfeffer, the tech lead of the Linaro Android working group (He’s always entertaining). Zach also has another very interesting presentation where he explains how to port Android to a new platform with Linaro.

If you want to learn more about Android internals, and use Android in embedded systems, check out Opersys “Android Variants, Hacks, Tricks and Resources” presentation slides (AnDevCon II) as well as their “Embedded Android Workshop” slides.

Training Materials about Embedded Linux and Android – Top

In this section, I’ll list some external resources and I (and many others) find very good to get started with embedded Linux or/and embedded Android.

Free Electrons is a French technology company offering embedded Linux consulting services and training, abd there have excellent embedded Linux training material about Linux kernel and driver development (labs.pdf), embedded Linux using IGEPv2 board (TI OMAP3) and a more than 500 pages presentation about embedded Linux system development (slides.pdf). The PDF downloads may not be the latest versions, and the latest documents are available in their repository. Here’s how to build the documentation.

Opersys is another training company – You’ll find the owner,Karim J. Yaghmour, gives presentation at nearly all Linux and Android events – that specializes in embedded software development courses. They offer their embedded Android and Linux materials for free.

The Linux foundation also provides free Linux training videos, although most are not directly related to Embedded software, there are still interesting Have a look at those free Linux tutorials.

Every year, there are events such as the Embedded Linux Conference and the Linux Conference, and videos are recorded (usually by Free Electrons mentioned above) and made available freely. I find those videos very informative, and by watching them you’ll learn many of the latest Embedded Linux, open source and Linux developments. I’ve uploaded a few to YouTube, written summaries and provided links to presentation slides if available:

The Embedded Linux Quick Start Guide presentation by Chris Simmons at ELCE 2010 is one of the most popular, and within three hours will walk you maybe aspect of embedded software development, with the last section being an hands-on session with NXP LPC3250 Stick (ARM9). Please note the second video is very noisy until 17:00 because of renovation being done at the time.

NPTEL has uploaded a 50-hour course about embedded systems by Dr.Santanu Chaudhury,Dept. of Electrical Engineering, IIT Delhi, on Youtube. This is not specific to embedded Linux, as it deals with all sorts of embedded systems, but it still seems very interesting.

The University of Edinburgh has made available their Embedded Systems Courses slides. Again this is not specific to Linux, although some parts are, but still interesting. Examples are using MQX RTOS and Freescale Kinetis K70.

Common Software Packages for Embedded Systems – Top

In “Big” Linux, when you want to install a web server you may think about Apache, a database you may think about Mysql, but those are generally way too resources intensive for embedded systems with limited memory and processing power.

So instead you’ll use lightweight alternatives:

ARM Programming – Top

This section is not specific to Linux and Android, but you may have to use this if you want to optimized ARM performance or the battery life of your products.

  • ARM development Cortex-A programmer guide – All you need to dig down at the register level and assembler level for ARM Cortex-A processor (Cortex A5, A7, A8 A9 and A15). The latest version 3.0 is available at https://silver.arm.com/download/download.tm?pv=1296010 but requires free registration.
  • ARM Neon Tutorial in C and Assembler – Presented by Doulos, another training company.
  • Compile with ARM Thumb2 to reduce memory footprint and improve performance. (A bit old, but hopefully still relevant).

You can also visit ARM infocenter which a massive amount of information on ARM programming.

Fixing Common Compile Error Messages – Top

When I browse some Linux or programming forums I often read people post “help” posts about errors during the configure and make stage of a build due to missing files or packages. A few years ago, It used to be that you had to do some research to find the right software package, but now thanks to packaging tools is very easy to find the missing software package and such questions should not show up in forums. I’ll give instructions using Ubuntu/Debian apt tools:

  • “fatal error: header.h: No such file or directory compilation terminated”. You can use apt-file to find the missing package.
  • “checking for PACKAGE… configure: error: Package requirements (package_name) were not met: No package ‘package_name’ found”. In that case you can use “aptitude search package_name” to get a list of packages containing “package_name”. and install the relevant one.
  • Finding configure option for a given package. Not an error as such, but it’s quite common that you have to cross-build a library yourself in embedded Linux as your system does not support packaging utils such as apt-get or yum, and may have to setup dependencies paths (headers and libraries) manually. A quick way to list those is to use pkg-config.

Programming Books – Top

Here are some links to programming books:

Debugging – Top

Here are some links to debugging tools, tips and how-tos to debug software in embedded systems:

Software QA – Top

In this section, I’ll write a bit about testing and list simple techniques to improve the source code by using some tools.

About 2 years ago, I wrote a post on how to share testing tasks between developers and testers when testing embedded software. this is for reference only, because other people will have a difference point of view.

Here are several resources, developers can do to improve code quality:

Finally, an interesting study found that to speed up development developers should use test automation, static analysis and virtual system prototyping/simulation (VSPS) tools, yet very few do.

Miscellaneous Tips and How-To – Top

Your application may need to support different languages, you can read the post entitled “Languages, Unicode and Charset” to learn why your app should use Unicode, how to convert different charset to unicode, and how to detect the language used in a file using tools and C code.

You should at least have basic code rules which indicate how to format your code, and many software can be setup to do that automatically. One of the rule is often to ban tabs in the code, so everybody uses (4) spaces instead of tabs to make the code well formatted for anybody. If you use vi/vim as your editor, you can set it to replace tabs by spaces automatically.

If you’re going to work on embedded Linux, you’ll have to use open source packages (one of them being the kernel), you should learn about the different open source licenses such as GPL, LGPL, Apache, MIT…

Here’s a method to redirect all output to a log file, it’s particularly useful for long builds.

You should really have documentation with your software, and you could write that separatly, but there are tools such as Doxygen which can generate the documentation automatically using variables, function names and specific comments in your source code.

Even though, with NAND flash it’s more common to use ext-2/3/4 nowadays, JFFS2 is still use in many embedded systems using NOR flash. You may need to access those images in a Linux PC to add files and/or see what’s inside. Here’s how to mount a JFFS2 image in Linux.

Where and How to Ask Questions – Top

Here’s a list of site useful to developers

  • stackoverflow – This is where you should ask questions related to programming
  • xda-developers forum – Excellent forum to find out how to hack mobile devices either to work with Android or Linux distributions. Information is all over of the place (due to the site popularity) and it may be difficult to find exact information, but I can tell you it’s there somewhere 🙂
  • Linux Questions forums – If you’ve got a problem with Linux, this is the place to ask. They also have section related to Embedded Linux, the Linux Kernel and hardware.
  • The board or silicon vendor – If you bought a development board, you should ask to your resellers, as many boards have online support. It should be the most obvious place to ask. Some companies such as Texas Instruments or Atmel have very active online communities, so you should ask in their forums to get answers from experts, instead of asking in more general websites such as Linux Questions.

If you think “Help me!”, “My board does not work!” or the grammatically incorrect ‘How do?” are good subjects for a forum post, I strongly recommend you read “How To Ask Questions The Smart Way“.

Before you ask a questions, make sure you’ve at least done some research with an online search tool such as “Google” (some people do not appear to know it exists in that case: https://lmgtfy.com/), provide an explicit title, explain what you’ve already tried yourself and give as much details as possible. If you have a long log, do not post it directly in the post, but post the relevant snippet, or if you are not sure where the problem is, copy/paste your log in pastebin.com and provide a link.

You are more likely to get an accurate answer that way, and you’ll also avoid ire from the community.

Let me know if you are more suggestions, find errors, obsolete instructions, etc.. in the comment section.

Radxa ROCK 5C (Lite) SBC with Rockchip RK3588 / RK3582 SoC

7 Replies to “Embedded Linux Development”

  3. Hi,
    I’m Ganesh from Bangalore,working as embedded software engineer.

    Before three month i bought LPC1788 development board from “http://www.hotmcu.com/hylpc1788-development-board-with-7-touch-screen-tft-lcd-p-39.html”. I tried to put uboot and uclinux on this so many times but couldn’t. I flashed uboot which downloaded from “http://www.emcraft.com/index.php/products/88” ,In this uboot i couldn’t load linux image through tftp . While browsing google i seen your blog profile. I’m struggling last three months So Please help me for this.

    Thank you.

    Reply

  5. hi,
    But in this CD Image , does’t have uboot and uclinux … but i need to load linux on that board..pls help me

    Reply

  6. @Ganesh
    OK. I see the kit does not come with Linux support. So you have to port Linux to the board. This is not a trivial task. Sorry I can’t help you.

    If you’ve already purchased Emcraft BSP, you could probably ask Emcrasf to help you.

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *

Boardcon Rockchip and Allwinner SoM and SBC products