KeK
Kindle environment Kit

Abstract

A general purpose collection of ARM tools for software development with some Kindle specific additions.
In addition to the collection of cross-platform development tools, utilizes the auFS file system to filter and maintain the development image in a task oriented, profile form.
The profiled image serves both the purpose of being a full distribution image for development and the purpose of providing a reduced size image for installation on the target machine.



1 Introduction

2 Image Profile

3 Scratchbox-2

4 QEMU

5 Web interface

6 Legal

6.1 This document

Copyright (c) Michael S. Zick, 2012, All rights reserved.
Licensed under the terms of the MIT license, included herein by reference: OSI, MIT License

6.2 Other works of authorship

Copyright is retained by the original author unless stated otherwise in a particular work of authorship.
Licensed under the terms of the MIT license, included herein by reference: OSI, MIT License unless stated otherwise in a particular work of authorship.

6.3 Community Contributions

Copyright is retained by the original author unless stated otherwise as a component of the contribution.
The project head is assigned as the non-exclusive agent of the Copyright owner.

6.3.1 New work

Licensed under the terms of the MIT license, included herein by reference: OSI, MIT License unless stated otherwise as a component of the contribution.

6.3.2 Components of existing work

License terms are those of the existing work to which the component contribution applies.

6.3.3 License terms unspecified

Works of authorship which do not specify license terms are licensed under the terms of the MIT license, included therein by reference: OSI, MIT License

6.4 Body of the MIT license

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

A Images in profile

A computer system implemented with a shared library system will have many runtime dependencies between the applications and the underlying libraries.
Linux uses a package management application to maintain a consistent runtime environment. Not all Linux distributions use the same package management application, but all complete distributions have one.
In the reduced resource world of Embedded Linux, the runtime environment will be made up of a selected subset of a complete distribution. Consistent and complete for the specific purpose the device serves but rarely complete from the view of a package management system.
A simple example: a Standards Complete system will include the complete set of documentation files, an embedded system will rarely contain the documentation files.
This situation leads to dual objectives for the system image: To keep the image easy to maintain and to keep the system small.
Meeting both objectives concurrently is a difficult task. The number of schemes to achieve that end is probably equal to the number of embedded build system designers.
One possible solution is to create a minimum dependency system distribution that remains small. This project includes an example of such a distribution (Ubuntu Core).
This project also includes an example of a Standards Complete, end user distribution.
The KeK project uses a layered file system (auFS) to maintain different file system views, with each view intended to meet one of the differing objectives.
This KeK, multi-view approach, will be applied to both types of the example systems.

A.1 Filtering basics

Pachinko
The main purpose of auFS is to provide a single, unified, view of multiple, populated, file systems. Each file system being what is termed a layer in this document. The total collection of unified layers is termed an auFS stack in this document.
The layers have an associated index number, starting from 0 as the Top with the Bottom having the largest index number. The Top, index value 0, layer should be writable. auFS will issue a warning if it is not writable.
When more than one writable layer is present in the auFS stack, there is a mount time option to control which of the writable layers is written during the creation and copy-up operations. Here, it is assumed that the default options: create=tdp cpup=tdp are in effect. Where tdp stands for: Top Down Parent.
The maximum number of layers supported in an auFS stack is a build time option. The default value is 127 layers.
When using directories of the same file system as the individual layers, no layer (directory) may be either an ancestor or a descendant of another layer. Here, we use parallel directories (all having the same parent) to avoid the case of having overlapped directory trees.
The system image filters built here are ordered layers with a few predefined directory paths. The mount-time ordering of the layers is based on the pathname components to match the default tdp behavior.
A short example of how the filtering action works, also showing how to capture all documentation in a single layer (single directory):
mkdir -p auroot This will be the mount point of our auFS stack.
mkdir -p wrklay This directory will be the index 0, catch-all layer.
mkdir -p usrlay/usr This layer will be the general catch-all layer for the /usr sub-tree of the fs image.
mkdir -p ushlay/usr/share This layer will be the general catch-all layer for the /usr/share sub-tree of the fs image.
mkdir -p doclay/usr/share/{doc,man,info} This layer will capture files from these three specific sub-trees of the fs image.
Create an auFS file system stack from these four layers (directories), mounted at auroot and given the arbitrary name of austack:
sudo mount -t aufs -o xino=/tmp/xino,noplink,\
br:wrklay=rw,\
br:usrlay=rw,\
br:ushlay=rw,\
br:doclay=rw \
austack auroot

Make the mount point of the auFS stack our current directory and, like firing a bucket of balls in a Pachinko machine, drop the contents of an existing file system image (say: deb7/*) onto it:
cd auroot
sudo cp -a ../deb7/* .

If we examined our current directory it will look the same as the contents of the directory we copied onto it. Because we did the copy operation as the super user, any special files and device nodes where copied.
Return to our common parent directory.
cd ..

While ignoring some hidden files (.wh.*), looking at the contents of the various layer directories shows the copied directory contents in profile (imagine looking at the side view of the stack).
Like the Pachinko machine, the predefined directory paths (pins) in the layers have captured selected file system sub-trees (balls). The most specific layer, the doclay directory, now has all of the image’s documentation.
If we where to remove the doclay layer from the stack, or take down the entire mount and put it back up as above without mention of the doclay layer, we would see that our goal of having a reduced image has been achieved.
For now, just unmount the stack:
sudo umount austack
In the course of a system’s development, the above process is always repeatable and fully reversible. The full view (all populated layers, in their original order) can be mounted and maintained with the common package management tools. The directories and files of the package management system would also be subject to the same treatment we gave the documentation. That is: they would not be included in the reduced image view.
The reduced view can also be created at any time, simply by mounting only selected layers.
In a real embedded system development case, there are many more layers involved than the few shown above to filter out all of the documentation.
The only trick here is in determining the predefined pathname contents of the various filter layers. In other words, the pin positions of this system image Pachinko machine.
Note: This analogy fails in that files not captured remain on the top (index 0) layer. In a Pachinko machine the balls not captured fall to the bottom.

A.2 Layer planning

The initial filter prototypes where created by hand, using a spreadsheet as one large table.
Tedious, but not as difficult as it might first appear, since most Linux file systems follow the Linux FHS.
The current work on the FHS is found in the FHS 3.0 Draft, the first revision since 2004.
Ultimately, the filter layer definitions depend on the file system view(s) the end user wishes to achieve. These details of these initial filter prototypes are expected to evolve over time.
Due to the volatile nature of these definitions, the original spreadsheets are placed under source control. Once the representation used in the spreadsheets is refined, a text version will also be placed under source control to allow the source control management to track the changes.

A.2.1 Creating the master list

Assume that a sample of the file system image exists, rooted in the directory name: image_root:
cd image_root
sudo find . -type d | sort >../image_root.csv
cd ..

The resulting file will list the directory paths in tdp (Top Down Parent) order.
A small excerpt:
./lib 
./lib/arm-linux-gnueabi 
./lib/arm-linux-gnueabi/security 
./lib/firmware 
./lib/init 
./lib/lsb 
./lib/modules 

A.2.2 The planning work sheet

This list can then be imported into a work sheet as the reference path in the first column.
The first entry (.) will become the general purpose catch-all layer (wrklay in these examples) at layer index 0.
Use the second and third columns of the work sheet to record your decisions.
The second column will hold either an indicator that this path defaults to its most specific parent or the numeric index of the layer with this particular pathname predefined.
The third column will hold the name (liblay for ./lib above) of the directory that represents the layer’s sub-tree. This name will be first used where an index number appears in column two, otherwise the name should already have been assign to a layer at a lower, numerical index.
The table resulting from the above small excerpt:
Predefined pathname Index Layer
./lib [12] liblay
./lib/arm-linux-gnueabi [*] liblay
./lib/arm-linux-gnueabi/security [*] liblay
./lib/firmware [13] firlay
./lib/init [*] liblay
./lib/lsb [*] liblay
./lib/modules [14] modlay
Everything under the ./lib sub-tree that isn’t more specifically qualified will end up in the liblay directory.
The firmware loaded by kernel drivers will end up in the firlay directory.
The kernel driver modules will end up in the modlay directory.
This small excerpt also demonstrates that the layer decisions in the prototype filters are subject to refinement.
In the case where the embedded system need not host a compiler, those related ./lib paths could have been put on a (removable) layer of their own.
In the case where both the kernel driver firmware and the kernel modules are both always shipped with the kernel, they could have been put on the same layer ([13]) with a: mkdir -p firlay/lib/{firmware,modules} define.
Which is an example of how the information in this decision table will be used.

A.3 Layer creation

Creation of the set of layers as a set of parallel directories in the prototype filters is done by a simple command file.
The creation is triggered by sourcing the command file from the command line after navigating to the desired parent directory.
Sufficient at this proof of concept stage in the project but such an implementation does not have much future in actual practice.
The algorithm used to create the commands from the work sheet table is simple:
For any row that has an index number in the second column, replace the leading period of the pathname in the first column with the directory name in the third column. The resultant string is the argument to the mkdir -p command in the command file.
For example, the work sheet row:
Predefined pathname Index Layer
./lib [12] liblay
Becomes the command line entry:
mkdir -p liblay/lib
When reading the command file prototypes, examples of the use of multiple command syntax will be seen. That is just a matter of style, not a component of the process.
The command file to create the auFS stack mount command can also be created directly from the information in the work sheet table.

A.4 Profile stack

Creation of the auFS stack from the set of layers (directories) is done by a simple command file.
Sufficient at this proof of concept stage in the project, but such an implementation does not have much future in actual practice.
The prototype command file contains a first line and a last line with some hard coded information that makes each one very specific. Good enough for a proof of concept prototype but should be implemented in a much more general manner in actual practice.
The first line of the multi-line command contains the privilege escalation and basic portions of the auFS mount command:
sudo mount -t aufs -o xino=/tmp/xino,noplink,\ 
The last line of the multi-line command contains an arbitrary name assigned to this instance of an auFS file system and the mount point name:
austack auroot
Each directory from the work sheet is listed in increasing index order. It is the order of listing in the mount command options which defines the layer’s index. Each layer is specifically assigned the read/write property. An excerpt from one of the command files:
sudo mount -t aufs -o xino=/tmp/xino,noplink,\ 
br:wrklay=rw,\ 
br:syslay=rw,\ 
br:etclay=rw,\ 
* * * *
br:vlblay=rw,\ 
br:vlalay=rw,\ 
br:vldlay=rw \ 
austack auroot 
The multiple lines make up a single, comma separated, list of mount options without any whitespace other than the last option entry where the whitespace ends the option list.
Since the auFS stack is a nodev type of file system, an arbitrary name, austack, is given to this instance and the mount point is named as the directory auroot relative to the current directory.

A.5 Future plans

The creation “by hand” of the filters and profiles described in this section is on life-support.
The replacement will be a Lua “executable configuration file” that should make life of the end-user a bit sweeter.

B Resources

The KeK project server mirrors a number of resource files in support of this document.
All files on that server have an associated md5sum file.

B.1 Cross compilers

The cross-compiler tool-chains (x86/Linux - arm/Linux, x86/Windows - arm/Linux) resourced are mirrored copies from their original publisher.
The two publishers represented, Mentor Graphics (previously known as Code Sorcery) and Canonical Ltd. (Ubuntu) ship cross-compilers that are well tested and used commercially. These resource postings include the posting of the sources used to create the binary tool-chains.
In the case of the Linaro (Canonical/Ubuntu) tool-chains, the tool-chain build files (crosstool-ng) are also posted. The posted crosstool-ng build system may be used to either duplicate the build of the binaries posted or to build the tool-chain based on glibc, uClibc or newlib (stand alone, non-hosted).
All of these tool-chains are built to be location independent and follow naming conventions that allow all of them to be installed on a machine at the same time.

B.1.1 Historic tool-chains

Provided for the rare case that a tool-chain closely matching the one used by Amazon/lab126 is required. The first two listed differ only in patch level.
arm-2006q3-26-* A close match for the K3, version 3.2.1 firmware, tool-chain. Probably also usable as a match to any of the i.MX3* SoC firmwares (K1, K2, KDX, K3).
arm-2010.09-50-* A close match for the K5, version 5.1.0 firmware, tool-chain. Probably also usable as a match for any of the i.MX5* SoC firmwares (K4, K5). A README file is available.
arm-2011.09-70-* The previously most recent public MG/CS release. The public releases typically lag the commercial releases by six months but have been in-use by paying customers during those six months. A README file is available.

B.1.2 MG tool-chains

As of the date of this document, the most current Mentor Graphics public releases. All gcc-4.6 compiler series based tool-chains. The listing of two source code packages is because MG/CS normally includes the scripting used to build the compiler release in the source code archive.
All of the public releases can be found on the Mentor Graphics release page and are down loadable after registration.
The self-installer bundles are not re-distributable, you have to get those from the above Mentor Graphics site. The bundles mirrored here are the re-distributable archives.
Hosted: Generates code that will be hosted by an operating system (in this case, Linux).
Standalone: Generates code that normally runs before an operating system is loaded (boot loaders, diagnostics, etc.)
Hosted (glibc):
getting-started-2012.03-57.pdfContents, changes and release notes
arm-2012.03-57-arm-none-linux-gnueabi.src.tar.bz2Tool-chain source code
arm-2012.03-57-arm-none-linux-gnueabi-i686-pc-linux-gnu.tar.bz232-bit Linux binaries
arm-2012.03-57-arm-none-linux-gnueabi-i686-mingw32.tar.bz232-bit Windows binaries
Standalone (newlib):
getting-started-2012.03-56.pdfContents, changes and release notes
arm-2012.03-56-arm-none-eabi.src.tar.bz2Tool-chain source code
arm-2012.03-56-arm-none-eabi-i686-pc-linux-gnu.tar.bz232-bit Linux binaries
arm-2012.03-56-arm-none-eabi-i686-mingw32.tar.bz232-bit Windows binaries

B.1.3 KeK tool-chains

For work that does not require compatibility with the Kindle firmware builds, the Linaro (Ubuntu embedded arm) tool-chains are preferred.
At this time, the tool-chain is based on gcc-4.7 series compilers and have additional run-time files to be used on systems built with gcc-4.6 series compilers. The run-times are dual-lib, multi-arch for both armel and armhf.
The project that is producing these tool-chains practices a monthly release cycle. The KeK project repository will probably not include mirrors of every release. The publisher's release page lists the releases as they are made.
gcc-linaro-arm-linux-gnueabihf-2012.05-* gcc-4.7 based tool-chain, default code generation is Cortex-A, Thumb-2, hard float, VFPv3-D16 FPU (neon) built against eglibc. A README file is available.
crosstool-ng-linaro-1.13.1-2012.05-* The crosstool-ng build system files used to create the tool-chain.
This collection includes both an archive version and a self-installer version for the Windows OS.
gcc-linaro-arm-linux-gnueabihf-2012.05-20120523_src.tar.bz2 Source files of the tool-chain.
gcc-linaro-arm-linux-gnueabihf-2012.05-20120523_linux.tar.bz2 32bit Linux binaries.
gcc-linaro-arm-linux-gnueabihf-2012.05-20120523_win32.zip 32bit Windows binaries (archive).
gcc-linaro-arm-linux-gnueabihf-2012.05-20120523_win32.exe 32bit Widows binaries (self-installer).
gcc-linaro-arm-linux-gnueabihf-2012.05-20120523_runtime.tar.bz2 gcc-4.7 target runtime files.
crosstool-ng-linaro-1.13.1-2012.05-20120523.tar.bz2 Build system files.

B.2 Integrated Development Environment (IDE)

The Eclipse-CDT IDE is a specialization of the Eclipse IDE for C/C++ developers.
Yes, this one also does Java development.
http://www.eclipse.org/cdt/downloads.php Home page.
http://www.eclipse.org/legal/epl/notice.php End user license agreement.
http://www.eclipse.org/downloads/packages/node/818 Release page, complete Eclipse CDT.
http://download.eclipse.org/tools/cdt/releases/juno P2 repository, for adding CDT to an existing Eclipse installation.
All of the mirrored file bundles are subject to the above EUL agreement. All are the Juno (v8.1.0) release. All require Java to be installed.
eclipse-cpp-juno-win32.zipWindows, 32-bit
eclipse-cpp-juno-win32-x86_64.zipWindows, 64-bit
eclipse-cpp-juno-macosx-cocoa.tar.gzMacOSx, 32-bit (cocoa)
eclipse-cpp-juno-macosx-cocoa-x86_64.tar.gzMacOSx, 64-bit (cocoa)
eclipse-cpp-juno-linux-gtk.tar.gzLinux, 32-bit (gtk)
eclipse-cpp-juno-linux-gtk-x86_64.tar.gzLinux, 64-bit (gtk)

B.3 Base file system images

The provided base file system images are selected to be ones of general interest.
deb7-buildd-armel The buildd variant of the Debian/Wheezy install as built by debootstrap from the main Debian repository (not the reduced emdebian). (ARMv5T, gcc-4.6, glibc)
The Ubuntu Core project is targeted at embedded Linux with a reduced size base install with minimized dependences.
The project description page provides an introduction to the project and the project goals.
The project release page is the source of the file bundles mirrored in this project.
The current UC rootfs images where built with gcc-4.6 and are expected to transition to gcc-4.7 in the near future. Until that transition, if using the gcc-4.7 based Linaro tool-chain, the Linaro provided run-time files for gcc-4.7 (listed above) should be installed in these images.
ubuntu-core-12.04-core-armel The reduced size Ubuntu/Precise base install targeted at the embedded Linux market. (ARMv5T, gcc-4.6, eglibc)
ubuntu-core-12.04-core-armhf The same reduced size Ubuntu/Precise base install as above, only built as “Hard Float”.

B.4 Filter and Profile sets

The filter and profile sets include the command files associated with them as, as-built snapshots of the command file development.
The most current version of the command files can be found in the KeK project repository .
filter-0.1-deb7-buildd.tgz The empty file system filter layers for the deb7-buildd-armel file system image.
profile-0.1-deb7-buildd.tgz The set of populated layers of the deb7-buildd-armel file system image.
filter-0.1-bun12.04-core-armel.tgz The empty file system filter layers for the ubuntu-core-12.04-core-armel file system image.
profile-0.1-bun12.04-core-armel.tgz The set of populated layers of the ubuntu-core-12.04-core-armel file system image.
filter-0.1-bun12.04-core-armhf.tgz The empty file system filter layers for the ubuntu-core-12.04-core-armhf file system image.
profile-0.1-bun12.04-core-armhf.tgz The set of populated layers of the ubuntu-core-12.04-core-armhf file system image.

B.5 Command files

The most current version of the command files and the filter/profile planning sheets can be found in the KeK project repository .

C Kindle specific

The device image files shipped with the Kindle devices are run-time system images, not complete for the purposes of development.
The filter/profile layering scheme is extended with additional layers to complete the file system images for development purposes without changing the factory released images.

C.1 Image model, v-3.2.1

C.2 Image model, v-5.1.0