Customizing Android application settings and application startup behaviour using overlays

When developing an Android custom device for a particular market you would need to customize the system application settings and behaviour such as enabling NFC after boot up or disabling the lock screen for testing etc.

For the above use case it is a bad practice to modify the settings or the code in the source tree of the application. The recommended way to change the settings is in something called the overlay. Overlay concept is similar to the ".rc" files present in Linux.

For example to change the default settings in the SettingsProvider  we have to change the settings in the defaults.xml. In the source tree of Android this package is present in frameworks/base/packages/SettingsProvider/res/values/defaults.xml

In this file you can find a lot of default settings under the resources tag such as def_dim_screen or def_accelerometer_rotation which enables the rotation of the screen by default etc.

To change a particular setting we have to create the similar folder structure in the overlays directory.
To do this we navigate to device/<vendor>/<product>/overlay. If there is no overlay directory create the directory.

Next we create the folder tree structure similar to the package folder structure mentioned above. So the final structure will be
device/<vendor>/<product>/overlay/frameworks/base/packages/SettingsProvider/res/value/

Next we create the defaults.xml file and copy only the needed settings inside the resources tag.

What the above does is overrides the settings present in the defaults.xml file in the application directory with the settings present in the defaults.xml file in the overlays directory.

Similarly we can create overlays for other applications or the frameworks. We can put the config.xml or the values.xml or even the wallpapers customized for the product.

Adding an apt repository from behind a proxy

Generally adding an apt repository from behind a proxy does not go well i.e. it ends with a "end of file". The reason being that the http_proxy and the https_proxy environment variables are not preserved in the sudo's environment.

To fix this we need to edit the sudoers file to have the following lines:

1 2	Defaults env_keep+="http_proxy" Defaults env_keep+="https_proxy"

Please remember to separate the "Defaults" and "env_keep" with a tab.

In this way the http_proxy and https_proxy will be preserved for the add-apt-repository to fetch the keys and the repo info.

Manual setting up of adb keys in Android

There are many cases where we would like to enable ADB debugging without the use of accepting the fingerprint in the UI especially in the cases where you are bringing up the display for embedded device or on a headless android.

If you have already used adb before then there would be ".android" folder present in Linux. If it is present then you can simply copy the adb_keys.pub to the folder in /data/misc/adb folder. 

If you have created a separate "data" partition then mount the partition

For eg:

mount /dev/sdc4 /mnt/android_mnt

After the mount just copy the file to /mnt/android_mnt/misc/adb.
In my case the permissions for the file is as follows:

-rw-r--r-- root     shell         717 2017-02-18 12:43 adb_keys 

If you do not have .android folder or the above keys then create the .android folder. To create the keys do a:

adb keygen adbkey

This will create the above 2 files adbkey and adbkey.pub. Copy the public key. Please note that the file created is adbkey.pub but the file name has to be adb_keys.pub when put in the misc/adb folder.

Pidgin setup for Microsoft Communicator/Lync/Skype

Let me break the hiatus by writing a small post on the setup of pidgin for MS Communicator/Skype. This applies to my network and might vary in your case. Internet resources for setup does not somewhat give a complete picture. Hence I decided to do a writeup.

The base data is that the network uses Skype for Business and Office 365. Hence we need to do the following:

1. Download pidgin using apt.
2. Download pidgin-sipe package using apt.

Next start up pidgin and do the following settings:

1. In the Accounts window (Accounts -> Manage Accounts)  select Add.
2. Go to "Basic" tab
3. Select "Office Communicator" as the protocol.
4. For user name enter it in the format of the email address of your office 365 account.
5. For "Login" field I enter in the format of "DOMAIN\user". Here "user" being the ldap login user name for your office 365 account.
6. Enter your password in the password field and check the box "Remember password"
7. Go to "Advanced" tab.
8. Enter "Auto" in connection type.
9. I have entered the User agent string as : "UCCAPI/16.0.6001.1073 OC/16.0.6001.1073 (Skype for Business)" minus the quotes. If you get an error please try other User agent strings as recommended in the FAQ given here: https://sourceforge.net/p/sipe/wiki/Frequently%20Asked%20Questions/#connection-refused-with-error-messagewzxhzdk12you-are-currently-not-using-the-recommended-version-of-the-clientwzxhzdk13you-have-been-rejected-by-the-server-httpsportalmicrosoftonlinecomdownloadlyncaspx
10. Select the Authentication Scheme as "NTLM". It might vary for your network. Please try different options if the one I selected does not work.
11. Keep all other options as defaults.
12. In the proxy tab select "Environment Settings".
13. Enable the account in the Accounts tab.
14. Restart pidgin if needed. 

 Enjoy communication with your colleagues.

If people do not believe that mathematics is simple, it is only because they do not realize how complicated life is.

-John Von Neumann

Slow updates

In the past few months life has taken a tough turn. It keeps getting harder and harder to concentrate on anything when mental peace is elusive. I have decided to slow things down a little bit and take things slowly. Hopefully I will be able to get back fresh and continue with my passion and thereby educate you, the viewers with my blog posts. You can always mail or comment on my posts which will help me get me strength.

So wish me luck!

Linux dts compilation using dtc compiler.

I was reading up on the dts (Device Tree Specification) specification and wanted to try out custom dts and compile it to dtb (Device Tree Blob).

The dts I wanted to compile was for the Wandboard which is the imx6dl-wandboard.dts

To compile I use the following command:

dtc -I dts -O dtb -o imx6dl-wandboard_test.dts

When trying this I get the error "FATAL ERROR: Unable to parse input tree near the includes..."

Turns out we have to use CPP (C Preprocessor)  to parse the includes and substitutions. Hence the next command is as follows:

cpp -nostdinc -I include -undef, -x assembler-with-cpp imx6dl-wandboard.dts > imx6dl-wandboard_bare.dts

 Here we use

-nostdinc:  We do not search system directories for header files. Only the directories I have specified wtih the -I are searched for header files.

-undef: We do not perform any system-specific or GCC-specific macros. All standard predefined macros remain definied.

-x: We provide the option assembler-with-cpp as the dtc compiler cannot auto ditacte whether the file is a "open", "clode", "read and write"

Pass the dts file and redirect the output to an output dts file.
Next we run the dtc compiler as above replacing imx6dl-wandboard_test.dts with the output generated from CPP. This will create the dtb file . You reverse the dtb to a dts by replacing the -I parameter with dtb and -O parameter with dts.

Also make sure you run this command in the arch/arm/boot/dts directory because of the includes present in it. Replace the "arm" with your CPU architecture of choice.

Additionally you can also create the dtb file running the rule dtbs in the makefile.

To make it work edit the Makefile in the arch/arm/boot/dts and add your dts file in the proper architecture. For my wandboard it will be dtb-$(CONFIG_ARCH_MXC).

Next create the dtb file with the following command:

1	ARCH=arm CROSS_COMPILE=`pwd`/prebuilts/gcc/linux-x86/arm/arm-linux-androideabi-4.9/bin/arm-linux-androideabi- make -C kernel_imx/ dtbs

References: https://linux-sunxi.org/Device_Tree#Compiling_the_Device_Tree

"There must be no barriers to freedom of inquiry … There is no place for dogma in science. The scientist is free, and must be free to ask any question, to doubt any assertion, to seek for any evidence, to correct any errors."

--J Robert Oppenheimer

Running bootchart in Android Marshmallow

For the Wandboard that I was customizing I wanted to know the process statistics, processor usage and IO usage to speed up my boot up time. I decided it was time to run bootchart.

There is quite a bit of outdated Internet resources on how to use bootchart for Android. I decided to write one:

First enable bootchart by doing a adb shell "echo 120 > /data/bootchart/start". This would tell the init to start bootcharting for 2 minutes. After this you get a set of files in the /data/bootchart directory. The files are generally as follows:

header
kernel_pacct
proc_diskstats.log
proc_ps.log
proc_stat.log

Next run the shell program grab-bootchart.sh in system/core/init which will use the measurement files above into a visual file either .png, .swf which shows the different processes hogging the CPU and which processes take a long time to init. Before we conclude a boot chart below of my wandboard custom build (Click for a larger image or download it to zoom in).

Anyone who considers arithmetical methods of producing random digits is, of course, in a state of sin.

 -Jon Von Neumann