Nvidia Shield - Android Graphics limitations for Developers (OpenGL ES 3.0)

   One of the most annoying problem for third party developers and platform providers is inability to use the available feature sets of the platform for a rich user experience. This is the same problem that Steve jobs claimed to be the case for Adobe's Flash. Adobe was slow to update Flash to use the latest features of Mac OSX and wasn't really helping third party developers. Besides, flash being closed source too didn't help.

    This was the case with Nvidia's shield platform too. Shield had Tegra 4 which supported OpenGL ES 3.0 APIs but wasn't really compliant to the standard. At launch, it was shipped with Android's Jelly Bean (4.2.1) which supported just Open GL ES 2.0. In this case, Android graphics APIs had become the bottleneck on shield's hardware capabilities. Nvidia could have launched an SDK Add-on exposing new APIs to use ES 3.0 but took safe route hoping quick adoption of latest Open GL standards by Android and yes it worked out fine. Android's 4.3 was the first version supporting OpenGL ES 3.0 and quick upgrade by Nvidia enabled third party developers to exploit all features.

Quick and easy setup of Database in Java

    Just ran into a need of a quick setup of database in Java and didn't want to spend time on File and String manipulation, fortunately came across Apache's Derby database and was able to get going within few hours and had enough time to get to other logic aspects. Started off using in-memory database and eventually moved onto a database hosted on a server without changing any of the application code.

    This implementation is based on derby jars (http://db.apache.org/derby/releases/release-10.10.1.1.cgi), derby.jar is for in-memory database and derbyclient.jar is for hosting a database as a service.

public class Database {

    Connection mConnection;

    String         mDbName;

    boolean      mAutoCommit;

    TYPE          mDatabaseType;

   

    public enum TYPE {

        IN_MEMORY,

        LOCAL_SERVER

    }

   

    public Database(String name, boolean autoCommit, TYPE databaseType) throws SQLException, ClassNotFoundException, TypeNotSupportedException, InstantiationException, IllegalAccessException

 {

        mDbName = name;

        mAutoCommit = autoCommit;

        mDatabaseType = databaseType;

       

        if (mDatabaseType == TYPE.IN_MEMORY)

        {

            // load JDBC Drivers for Java DB (In-memory database)

            String className = "org.apache.derby.jdbc.EmbeddedDriver";

            Class.forName(className).newInstance();

           

            // establish an active connection to the database

            mConnection = DriverManager.getConnection( "jdbc:derby:memory:" + mDbName +";create=true" );

        }

        else if (mDatabaseType == TYPE.LOCAL_SERVER)

        {

            // load JDBC Drivers for Java Client DB (database)

            Class.forName("org.apache.derby.jdbc.ClientDriver").newInstance();

           

            // establish an active connection to the database

            mConnection = DriverManager.getConnection( "jdbc:derby://localhost:1527/" + mDbName + ";create=true" );

        }

    }

   

    // Create, update tables

    public void executeDDLStatement(String query) throws SQLException 

   {

        Statement s = mConnection.createStatement();

        s.execute(query);

    }

   

    // Select queries

    public ResultSet executeQueryStatement(String query) throws SQLException

   {

        Statement s = mConnection.createStatement(ResultSet.TYPE_SCROLL_INSENSITIVE, ResultSet.CONCUR_READ_ONLY);

        ResultSet rs = s.executeQuery(query);

       

        return rs;

    }

   

    // Update, delete queries

    public void executeUpdateStatement(String query) throws SQLException

   {

        Statement s = mConnection.createStatement();

        s.executeUpdate(query);

    }

   

    public void commit() throws SQLException {

        mConnection.commit();    

    }

   

    public void rollback() throws SQLException {

        mConnection.rollback();

    }

   

    public void close() throws SQLException {

        mConnection.close();

    }

   

  When using derbyclient, the database server runs by default on port 1527 and can be customized and if connections should be allowed from localhost and any other server use the following.

          ...\db-derby-10.10.1.1-bin\bin\startNetworkServer -h 0.0.0.0
   

Android - Messenger vs Binder IPC

     Android's Messenger pattern is an abstraction on top of binder inter process communication to quickly implement a client-server interface. It can be used for both inter and intra process boundaries as the fundamental link is the binder driver.  Developers don't even have to deal with AIDL and instead just have to deal with an application level protocol between the interacting components and define messages. However, Messenger might not always fit the needs as,

     All messages are routed through a single thread which creates the handler for the Messenger. Although, this has the advantage of thread safety, it could also mean potential delays, especially when multiple requests could be serviced simultaneously. Besides, there is also a delay for non simultaneous requests when using Messenger.

   And, none of the requests from a client could be blocking unlike a AIDL interface api, which could either be one way or blocking. Server needs another Message to send its response. This is because of Messenger's design over binder.

IMessenger.aidl

          oneway interface IMessenger {

               void send(in Message msg);

          }

  The Message is nothing but a Parcelable which is actually sent across IMessenger binder interface as a one way transaction. The request is processed in the context of a binder thread on the server end and is relayed to the message queue of the thread which created the Messenger's handler.

Handler.java

          private final class MessengerImpl extends IMessenger.Stub {

                public void send(Message msg) {

                       Handler.this.sendMessage(msg);

                }

          }

    This relaying of the message from the binder thread to the handler thread is what causes the delay and makes all interactions as non-blocking when using Messenger pattern.
    

Android - Foolproof low memory killer

    Android's low memory killer is based on application's out of memory score (oom_adj). The source of truth is in a file in proc filesystem (/proc/pid/oom_adj). Although, this is not really specific to Android, it is effectively used by Activity manager to help determine important applications. A foreground application being the most important application gets a score of 0. Higher the score, lesser is the importance and higher probability of being killed by low memory killer.

  3rd party application developers might look for a way to cheat low memory killer and protect their app from being killed. However, this is not possible, not because the file /proc/pid/oom_adj can't be accessed or written into, instead is because of the sys call corresponding to a write request on the file.

   An application code might look like this,

        String fileName = "/proc/" + android.os.Process.myPid() + "/oom_adj";
        File oomFile = new File(fileName);

         try {
                FileOutputStream os = new FileOutputStream(oomFile);
                final String value = "4";
                os.write(value.getBytes());
                os.close();
         } catch (Exception e) {
                throw new RuntimeException("Unable to update score [" + e.getMessage() + "]");
         }

   Now this code can work as well as fail depending upon the current state of the application or to be more specific the current oom score. If the application is in foreground and if the above code is executed, it would indeed update the oom_score to 4. Upon closer look, it actually updates it from 0 to 4, a potential degrade and higher probability to be killed, a potential suicide.

   So the next alternative is to run this code from a service, which puts the application in background and normally tends to give the process a non-zero oom score. So the tendency would be to set 0 in this case. However, the code fails with a IOException saying access denied (EACCESS).

   How can a FileOutputStream API know about the current foreground status of the application and fail selectivity? Actually it doesn't and the code fails on any attempt to set a lower score than the current score, irrespective of the status of the application.

   Turns out this failure is actually triggered by the sys call implementation of POSIX write() for oom, oom_adj_write. Access is denied when the current process isn't capable of updating and requests a score lesser than the current score. This also explains as to how Android framework gets to update it for applications, just being capable of doing so. :-)

   Even though applications can't directly lower the score, Toast API offers an exploit to lower oom_score documented here.

Android Shared User ID and android:process

      Android's shared user id (android:sharedUserId) is a way for different applications to share the same user id for permissions, which lets them access their data and host their components in other process space. By declaring a new sharedUserId in the manifest, a new user id is creating during installation and is ready to be used by other applications provided they are signed with the same certificate and one such use is to update android:process of an application component to host it in other process space.

   However, if the applications don't share a user id and aren't signed with the same certificate, a new process is launched with the same name as in android:process and this new process automatically shares userId with the requested application.

   If the applications share user id and and aren't signed by the same certificate, the installation of the apk would fail.

    If the applications share user id and are signed by the same certification, then requested process indeed hosts the component. There is an exception to this in latest version of android supporting global and application private process. This is documented here.

    If they don't share a user id and are signed by the same certificate, a new process is launched with the same name as in android:process and the new process shares userid with the requested application.

    However, there is one exception to the final rule in case of settings.apk and stk.apk. They don't share a user id but are signed with the same system certificate and yet requested settings's component is hosted in com.android.phone process.

Stk

       

                xmlns:androidprv="http://schemas.android.com/apk/prv/res/android"

                package="com.android.stk"

                android:sharedUserId="android.uid.phone">

                ...

                ...

                ...

       

                android:label="@string/app_name"

                android:clearTaskOnLaunch="true"

                android:process="com.android.phone">

Settings

       

                package="com.android.settings"

                coreApp="true"

                android:sharedUserId="android.uid.system">

       

                android:label="@string/sim_lock_settings"

                android:theme="@android:style/Theme.Holo.DialogWhenLarge"

                android:process="com.android.phone">

           

               

               

               

           

        

Android - Application's UID persistent storage

        Each package is given a unique user id during its installation and this remains the same as long as it stays installed in the device. The UID is generated by com.android.server.pm.Settings::newUserIdLPw(...) and is eventually saved in /data/system/packages.xml for persistent storage.

    

Android - context.stopService

Android's stopSevice's documentation suggests that the entire operation of stopping a service is synchronous and that the caller could use the result (boolean) to determine if the requested service was stopped or not. However, in my case, the requested service's onDestroy() wasn't invoked immediately, especially that it was hosted in its own process space. Turns out that the framework's callback to the requested service is actually one way

  mRemote.transact(SCHEDULE_STOP_SERVICE_TRANSACTION, data, null,
                          IBinder.FLAG_ONEWAY);

and the callback isn't handled immediately in the binder thread, instead if posted to the main thread which ends up invoking onDestroy() on the service.

 And as far is the caller is concerned, a result is returned even before the requested service receives the callback.

int stopServiceLocked(IApplicationThread caller, Intent service,
            String resolvedType, int userId) {
        ...
        ...
        ...
        ServiceLookupResult r = retrieveServiceLocked(service, resolvedType,
                Binder.getCallingPid(), Binder.getCallingUid(), userId, false, false);
        if (r != null) {
            if (r.record != null) {
                final long origId = Binder.clearCallingIdentity();
                try {
                    stopServiceLocked(r.record);
                } finally {
                    Binder.restoreCallingIdentity(origId);
                }
                return 1;
            }
            return -1;
        }

        return 0;
    }

Android - Active References to Finished activity

      The most annoying memory usage for an android application is active references to finished activity and yet this is one of the most missed use cases and to make it worse the activity in turn could hold references to other application components and data structures effectively making it similar to its foreground version (in terms of memory usage). As a thumb rule, the dalvik heap after garbage collection shouldn't have any instances of the finished activity.

      A simple test is to finish all application components like activity by pressing back key and initiate a garbage collection via DDMS and then look for leaks in the heap snapshot.

Android - Adb over Wifi and random disconnects

Android debug bridge being an application protocol can be run on top of different transport protocols. Most often used is over USB, however it can be run over WiFi. This could be used to quickly isolate random adb disconnect issues as the application layer protocol stays the same over all physical transports.

   * Make sure developer options is turned on and the host machine is authenticated for adb usage.
   * Connect the device via USB and request for the adb on the device to be restarted over TCP

                 adb tcpip 5555

   * Disconnect USB
   * Find the IP address of the device via Settings > About Phone > Network

             

   * Connect to the device over TCP via

                 adb connect ip_address:5555

   * Disconnect from previously connected device.

                 adb disconnect

 For folks running CyanogenMod, a developer options makes the setup quite simple. CM11 has this feature (ADB over network),