How to obtain the battery stats from an Android device

Source: https://developer.android.com/tools/performance/batterystats-battery-historian/index.html

Google provides an open-source Battery Historian Phython script that plots the power stats, but it is not necessary, especially if you intend to do customized processing on the stats file. Therefore, this post focus on the post-processing of the power stats, e.g., text processing, plotting, etc.

Battery data gathering

1. First, connect your device to your developer machine via USB. Check: bash $ adb devices List of devices attached R3AF600RD4Kdevice
2. Reset the battery readings. According to Google, > Resetting erases old battery collection data; otherewise, the output will be huge. bash $ adb shell dumpsys batterystats --reset
3. Disconnect the device, such that Android only draws current from the device battery.
4. Play with the app to be tested for a while. This should be long enough to incur certain amount of battery level changes.
5. Reconnect the device.
6. Dump the battery data. bash $ adb shell dumpsys batterystats > batterystats.txt Thus far, the data collection part has finished.

Data processing

Here are some of my scripts that process the battery stats. They are not very stable, so you might want to check the output at each step and make sure the processing was correct.

1. Obtain only battery info.

# getBatteryInfo.sh
grep '         ' batterystats.txt > batterylog.txt
sed -i 's/         //g' batterylog.txt
sed -i 's/(.)//g' batterylog.txt
sed -i 's/+//g' batterylog.txt
sed -i '1d' batterylog.txt
awk '{print $1,$2}' batterylog.txt > batterylog.txt.tmp
mv batterylog.txt.tmp batterylog.txt
dos2unix batterylog.txt
sed -i 's/ms//g' batterylog.txt
sed -i 's/m/:/g' batterylog.txt
sed -i 's/s/./g' batterylog.txt

1. Using R to continue processing and plot.

# plot.R
library(gdata)
options(stringsAsFactors = FALSE)
df=read.table('batterylog.txt')
timestamp=df$V1
battery=df$V2/100
options(digits.secs=3)
timestamp[grepl("\\.", timestamp)==FALSE]=paste("00.",timestamp[grepl("\\.", timestamp)==FALSE],sep='')
timestamp[grepl(":", timestamp)==FALSE]=paste("00:",timestamp[grepl(":", timestamp)==FALSE],sep='')
timestamp=strptime(timestamp, format="%M:%OS")
timestamp=timestamp-timestamp[1]
plot(timestamp, battery, type='s')

# calculate slope
model1=lm(formula = battery ~ timestamp)
#summary(model1)
#par(mfrow = c(2, 2),pty="s")
#plot(model1)
lines(timestamp, fitted(model1), type='o', col='red')

# calculate power
# spec of specific device
current=0.3 #amp
voltage=3.8 #volt
duration=3600 #second
capacity=current*voltage*duration #joule
rate=as.numeric(model1$coefficients[2])
power=-capacity*rate
power=round(power,4)
print(power)

How to upgrade Samsung Gear Live Watch

This post is about my journey on upgrading a Samsung Gear Live Watch purchased on Amazon. The OS was 5.0.1, and my goal was to upgrade its OS to Android 5.1.1. Perhaps because the onstock system image was a developer version, the default Update button does not give me an update. I tried many times to install a customized system image, but adb sideload filename.zip does not work (due to an error:closed error), forbidding me to load any external file onto the watch to install. Fianlly, I used Samsung Gear Live Restore Tool V2 from xda and reversed to a system image that allows automatic system upgrade and then upgraded from there. Here are the steps.

1. Search and download the restore tool.
2. Unzip, pre-process the files. Since the developer of this tool (a shell script called 2-Gear-Live-Tool-for-Linux-and-OSX.sh) used a Windows development environment, I had to use dos2unix 2-Gear-Live-Tool-for-Linux-and-OSX.sh to convert the bash script file to a Unix-friendly one. Also, the script calls a few binaries which are included in the same zip file. For convenience, I just did chmod +x * for everything in the folder so that the binaries will have execution permissions.
3. Bug. Due to some reason (perhaps dos2unix), one image file is mal-named in the script. Open the script and search for 78yrecovery, and change it to 78Yrecovery.
4. Boot the android watch into bootloader. This can be done either using adb reboot bootloader (if the watch's debugging interface is enabled) or reboot the device (by holding the power button) and swipe twice from top left to bottom right when the Samsung logo shows up (tip: this is easier to do when the power cable is disconnected).
5. Execute the shell script, and type restore. Follow the instructions in the script.

Only the V2 of the Restore Tool worked for me. After the restoration, the watch's OS reversed to Android 4.4W. Select System Update in the Settings and download/upgrade the system. The watch will then update to Android 4.4W-2. Repeat the process, update the system to 5.0.1 and then 5.1.1. Voila!

How to solve adb devices no permission error

Very often, when we connect a new device to the developer machine, we might encounter no permission error when we do adb devices, and even adb kill-server might not help. If the device's developer option is enabled, the error is most likely caused by the developer machine's udev settings. Here is how I solved it.

1. Connect the device to the machine via USB.
2. Check lsusb and find the entry related to the device. Look for keywords like Samsung or Google. One way to find out the usb info of the device is to disconnect and reconnect it and see the difference in the output. For example, here is a sample output without connecting:

$ lsusb
Bus 002 Device 004: ID 148f:5370 Ralink Technology, Corp. RT5370 Wireless Adapter
Bus 002 Device 003: ID 046d:0825 Logitech, Inc. Webcam C270
Bus 002 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 001 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 003 Device 003: ID 046d:c52b Logitech, Inc. Unifying Receiver
Bus 003 Device 002: ID 2516:0004  
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

And here is the output with the device connected:

$ lsusb
Bus 002 Device 004: ID 148f:5370 Ralink Technology, Corp. RT5370 Wireless Adapter
Bus 002 Device 003: ID 046d:0825 Logitech, Inc. Webcam C270
Bus 002 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 001 Device 020: ID 18d1:4ee7 Google Inc. 
Bus 001 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 003 Device 003: ID 046d:c52b Logitech, Inc. Unifying Receiver
Bus 003 Device 002: ID 2516:0004  
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

Observe the difference:

Bus 001 Device 020: ID 18d1:4ee7 Google Inc. 

Now edit the udev file at

$ sudo vi /etc/udev/rules.d/51-android.rules

And add this line:

# Samsung Gear Live
SUBSYSTEM=="usb", ATTR{idVendor}=="18d1", ATTR{idProduct}=="4ee7", MODE="0600", OWNER="user1"

where user1 is my username.

Restart udev:

$ sudo udevadm control --reload-rules
$ sudo service udev restart

Now we can find that the device can be properly recognized by adb (and a permission confirmation might prompt on the device).

$ adb devices
List of devices attached
R3AF600RD4Kdevice

How to install Mesos and Spark for a cluster

This post is a tutorial on installing Mesos and Spark frameworks on a cluster of servers.

1. Install Mesos [1]

Note: In this example we use master.com as the IP address of the master server. Remember to replace it to your own master server IP address. The bash scripts here require you to have sudo privilege for package installation and configuration. It is also assumed that hdfs is already setup in the cluster.

1.1 Set up Master

# mesos_master_setup.sh
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv E56151BF
DISTRO=$(lsb_release -is | tr '[:upper:]' '[:lower:]')
CODENAME=$(lsb_release -cs)
echo "deb http://repos.mesosphere.com/${DISTRO} ${CODENAME} main" | sudo tee /etc/apt/sources.list.d/mesosphere.list
sudo apt-get -y update
sudo apt-get -y install mesos marathon
sudo sh -c "echo 1 > /etc/zookeeper/conf/myid"
sudo sh -c "echo server.1=master.com:2888:3888 >> /etc/zookeeper/conf/zoo.cfg"
sudo service zookeeper restart
sudo sh -c "echo zk://master.com:2181/mesos > /etc/mesos/zk"
sudo service mesos-slave stop
sudo sh -c "echo manual > /etc/init/mesos-slave.override"
sudo service mesos-master restart

1.2 Set up Slave(s)

# mesos_slave_setup.sh
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv E56151BF
DISTRO=$(lsb_release -is | tr '[:upper:]' '[:lower:]')
CODENAME=$(lsb_release -cs)
echo "deb http://repos.mesosphere.com/${DISTRO} ${CODENAME} main" | sudo tee /etc/apt/sources.list.d/mesosphere.list
sudo apt-get -y update
sudo apt-get -y install mesos
sudo service zookeeper stop
sudo sh -c "echo manual > /etc/init/zookeeper.override"
sudo sh -c "echo zk://master.com:2181/mesos > /etc/mesos/zk"
sudo service mesos-master stop
sudo sh -c "echo manual > /etc/init/mesos-master.override"

1.3 Launch a slave

# mesos_slave_start.sh
sudo rm -f /tmp/mesos/meta/slaves/latest
sudo mesos-slave --master=master.com:5050 --hadoop_home=/usr/local/hadoop --hostname=$(hostname)

At this point, you are able to view the Mesos Web UI at master.com:5050.

2. Install Spark on top of Mesos [2]

2.1 Configuration

This step can be done in any server inside the cluster. You might need to log in as the hadoop user in order to upload Spark binary files onto hdfs.

# mesos_spark_setup.sh
su hduser1
wget http://www.apache.org/dyn/closer.lua/spark/spark-1.6.0/spark-1.6.0-bin-hadoop2.6.tgz
hadoop fs -put spark-1.6.0-bin-hadoop2.6.tgz / 
tar xvf spark-1.6.0-bin-hadoop2.6.tgz
cd spark-1.6.0-bin-hadoop2.6
hadoop fs -put ./lib/spark-examples-1.6.0-hadoop2.6.0.jar /

2.2 Launch Spark Dispatcher

./sbin/start-mesos-dispatcher.sh --master mesos://zk://master.com:2181/mesos

2.3 Submit a Spark Job to Mesos

./bin/spark-submit --class org.apache.spark.examples.SparkPi --master mesos://master.com:7078 --deploy-mode cluster  --supervise --executor-memory 20G --total-executor-cores 20 hdfs://master:54310/spark-examples-1.6.0-hadoop2.6.0.jar 50000

The specific port numbers might vary according to different system settings. If the job submission fails, check the error logs. Make sure that the firewalls do not block required ports.

2.4 Stop Spark Dispatcher

After your job finishes (which can be verified on the Web UI), you can stop the Spark job dispatcher. bash ./sbin/stop-mesos-dispatcher.sh