Thursday, May 17, 2012

Precompiled headers in vs-android

vs-android 0.93 does not have built-in support for precompiled headers, but it's possible to use precompiled headers with some small hacks.

According to the GCC documentation about precompiled headers, you're supposed to compile one like an ordinary source file. Presumably all you have to do is change the file extension from ".o" to ".h.gch" and GCC will pick it up as a precompiled header. The documentation implies you should put the gch in the same folder as the header, but I think that is a bad idea because Debug builds and Release builds must have separate gch files.

So instead what we will do is place the ".h.gch" file in the output folder with all the other ".o" files, and then add that folder with high priority so it is found before the real header file. Let's begin.
  1. Open your vs-android project file (*.vcxproj) in a text editor (instead of Visual Studio).
  2. Find the first <ClCompile Include="..."/> line. This line should be preceded by a <ItemGroup> opening tag.
  3. Assuming your precompiled header is called "Stdafx.h" and it is located in the "MyFolder" folder, insert the following code prior to <ItemGroup>
      <ItemGroup>
        <ClCompile Include="MyFolder\Stdafx.h">
          <AdditionalOptions Condition="'$(Configuration)|$(Platform)'=='Debug|Android'">-x c++-header %(AdditionalOptions)</AdditionalOptions>
          <AdditionalOptions Condition="'$(Configuration)|$(Platform)'=='Release|Android'">-x c++-header %(AdditionalOptions)</AdditionalOptions>
          <ObjectFileName Condition="'$(Configuration)|$(Platform)'=='Debug|Android'">$(IntDir)%(FileName).h.gch</ObjectFileName>
          <ObjectFileName Condition="'$(Configuration)|$(Platform)'=='Release|Android'">$(IntDir)%(FileName).h.gch</ObjectFileName>
        </ClCompile>
      </ItemGroup>
    
    It is important that the precompiled header goes into its own ItemGroup prior to the other ItemGroups in the project file, in order to control the build order. ItemGroups are built in the order they appear, and it's important that our precompiled header build first. However, VS/MSBuild/vs-android does not respect the order of source files within a single ItemGroup.
  4. Open your solution in Visual Studio.
  5. Right click the project and click Properties.
  6. Beside Configuration, choose All Configurations.
  7. Go to Configuration Properties | C/C++ | General and then edit the list of Additional Include Directories.
  8. Add $(IntDir) as the first directory.
  9. Build your project. It should build faster now, but there will be no additional output except that Stdafx.h will build before the other source files. In order to verify that build time decreased, you can disable building StdAfx.h and then Clean and Rebuild the project and use a stopwatch to see if it takes more time. To disable StdAfx.h, right-click StdAfx.h, choose Properties, click General, then set Excluded From Build = Yes. Remember to re-enable it afterward.
In my slowest-building project, build time was cut in half, from 3:52 to 1:58 for 48 source files.
Note to self, do not edit this post. Blogger will corrupt the angle brackets.

Android + NDK tips, part 3: the actual tips

Previous parts

Android development download checklist

General tips

  • Android programs don't have a specific entry point. The AndroidManifest.xml file specifies which Activity is started by your application icon in the launcher. But I forget how it works exactly.
  • Android programs don't have a specific exit event, either. The system can be terminate inactive programs at any time without notice. Read about the "Activity Lifecycle" in the documentation of the Activity class for more information. Each activity is supposed to save state whenever it is paused (e.g. the user clicks Home) and restore that state automatically when the activity is restarted.
  • Activities are created on a "stack", and the back button takes the top activity off the stack. I think I heard that if your app crashes with several activities on the stack, Android may restart the app with all activities except the last one. Visually it may appear that only one activity stopped, when in fact the entire process was stopped and restarted.
  • Android apps are fundamentally Java-based. I don't know any way to make a pure C/C++ program. Although it is possible to write a program without any Java source files, it will rely on Google code that accesses JNI internally and it will still be compiled to a *.so file, not an executable. The vast majority of functionality is only available through calls to Java, so it's best to just accept Java and live with it.
  • All Android applications are stored as a *.apk file, in the zip file format. Now, earlier I compared apk files to Windows Mobile cab files, but there is a big difference. Specifically, an apk is not unpacked! The program runs on Android "directly" from the apk (only being unpacked in memory, or perhaps to a temp folder, I don't know.) That's not to say that the apk does not have to be installed, though. Each Android app has its own user ID, and must somehow get its icon in the app launcher, so Android probably has some kind of installation process, I just don't know anything about it.
  • Android doesn't use a "real" Java Virtual Machine. It uses a Java alternative called Dalvik, which has its own instruction set (*.dex = Dalvik executable file), which is more compact than Java bytecode. Even so, it looks like a normal Java compiler is used and then its output, the *.class files, are converted afterward to Dalvik format prior to deployment on Android. The Dalvik JIT is said to run code more slowly than Oracle's hotspot JIT.

Eclipse

  • Don't like Eclipse key shortcuts? Use Window|Preferences, then General|Keys. For example, if you think F3 should find the next occurrance of your search text, type Find Next in the search box, click Binding and press F3. However, F3 is already used by some other, minor command. So you should probably search for F3 to find that other command and unbind it.
  • If Eclipse "intellisense" isn't working very well for you, reconfigure it in Window -> Preferences -> Java/Editor/Content Assist; see http://stackoverflow.com/questions/2943131/eclipse-intellisense
  • Eclipse is unaware of the NDK. However, once the NDK build is complete, Elipse automatically detects and deploys your *.so file without any instruction from you. However, when you rebuild your *.so file (outside Eclipse, presumably--I don't know how to build it inside Eclipse), Eclipse does not detect the new version automatically and therefore may not deploy it. To tell Eclipse about the new file, click the "libs" folder in Package Explorer and press F5 to Refresh before running your program (there is no visual indication that a Refresh occurred).

C/C++/NDK

  • There are two options for using C++ code:
    1. Build it with the command-line ndk-build script that comes with the NDK, then use Eclipse or another Android-supporting tool to build the Java code and deploy. This is the best approach if your program is mainly written in Java.
    2. Use a vs-android project with Visual Studio 2010 (Express is not supported). This is the best approach if your program is mainly written in C++, with minimal Java (vs-android has no features or IntelliSense for Java, and certainly no debugging support.)
  • Either way, you'll need to write JNI wrappers in C in order to call the C/C++ code from Java. (If you use Mono for Android, you can use P/Invoke to call C from C#.)
  • In C++, __android_log_print () can print messages to the LogCat pane in Eclipse. In Java, one-letter methods such as Log.w and Log.d do the same thing.
  • If a project uses a jni\Android.mk file, the C++ code is built on the command line by going into the jni\ folder and running the ndk-build script. ndk-build builds C++ only; you need to use Eclipse or some other tool to build the Java code and deploy the project.
  • I have been able to hook ndk-build into a Visual Studio 2008 project (WITHOUT vs-android) by creating a Custom Build Step that looks like this:
      SET NDK=C:\...\android-ndk-r7b
      SET SED=%NDK%\prebuilt\windows\bin\sed
      rem For some reason [0-9]+ does not work. Use [0-9][0-9]* instead.
      %NDK%\ndk-build -C C:\...\jni 2>&1 | %SED% "s#:\([0-9][0-9]*\):#(\1):#" | %SED% "s#//jni#/jni#"
    
  • I used the "sed" command to reformats the error messages slightly so that Visual Studio understands them. This allows you to double-click an error to view the correct location in the source code.

vs-android tips

  • vs-android doesn't come with the ability to create a new project. You must copy a sample project and modify it.
  • If a project uses a vs-android *.vcxproj file, you can open the project in Visual Studio 2010 and run it like any other C++ project. vs-android will build both the C++ code and the Java code. vs-android does not notice C++ source files automatically; you must add them to the project. However, it automatically picks up all Java source files from the "src" directory (recursively).
  • vs-android doesn't support the "Start without debugging" command (let alone "Start Debugging"). To restart a program that you already built, you must restart it from the Android device's launcher.
  • vs-android requires you to issue a three "setx" commands in order to configure it on your machine. There is one more command that the setup instructions don't tell you about; without this command Java may give an out-of-memory error. The command is: setx _JAVA_OPTIONS "-Xms256m -Xmx512m"
  • There is a small bug in vs-android (0.93) when configured to create output for the armv7-a architecture. The .so file goes to libs/armeabi (which is the folder for armv5te) instead of libs/armeabi-v7a. An ARM v7a device will prefer to load the .so from armeabi-v7a, but it will use libs/armeabi as a fallback.
  • Using precompiled headers in vs-android

Mono for Android

  • Currently, even programs written in C# using Mono for Android use JNI to access most of their functionality. Unfortunately, if you want to access Java classes from C# (classes that are not explicitly supported by Mono for Android) you must currently do so through the same clumsy JNI interface that C/C++ code must use.

Unsolved mysteries

  • I can't get Mono for Android to work. All of the sample programs crash on startup for me.
  • Eclipse: how to set the "startup project" that runs when you press Ctrl+F11?
  • Which SDK version is honored, this one in AndroidManifest.xml, or the one in project.properties or default.properties?
  • How to debug C/C++ code (details)?

Android + NDK tips, part 2: Android project directory structure

An Android project folder is pretty cluttered. Most of the files and directory structure are determined by Eclipse and Android conventions, and many of the files and folders are auto-generated so there is nothing you can do about the clutter.

Below I describe the folder structure of an Android project. Please note that the terms "auto-generated" and "generated" are not the same. Auto-generated files are generated automatically on-demand, so you can safely delete them and they need not be checked-in to source control. Generated files are created by tools, but you can't delete them because they will not be regenerated. Auto-generated folders may contain content that is not auto-generated, so they are not always safe to delete.
Directory name   Purpose
--------------   -------
src              Conventional location for Android Java source code*
jni              Conventional location for C/C++ source code**
res              Conventional location for Android XML resource files
res/layout       Conventional location for dialog layouts
res/values       Conventional location for strings (to support internationalization)
res/drawable     Conventional location for bitmaps and other drawables
res/drawable-*   Auto-generated by Eclipse to hold different bitmaps for different screen densities
assets           Auto-generated by Eclipse for files to store in your *.apk (use AssetManager to read)
bin              Auto-generated by Eclipse to hold the *.apk, Java *.class files and other output files
gen              Auto-generated by Eclipse or vs-android to hold generated Java files
libs             Auto-generated by ndk-build to hold lib*.so file(s)
obj              Auto-generated by ndk-build to hold C/C++ object files
* Java package and class names are required to match the directory structure. A class named com.foo.Thing must be located at src/com/foo/Thing.java
** However, C/C++ source code can be placed anywhere you want
File name               Purpose
---------               -------
.classpath              Unsure, may help find Java dependencies. Generated by Eclipse when making a project.
.project                Generated by Eclipse to hold the project's name.*
AndroidManifest.xml     Specifies App's icon, required Android version, and a list of all activities.
default.properties      Unknown (generated)**
project.properties      Unknown (generated)**
proguard.cfg            Unknown (generated). May be related to Java obfuscation***
local.properties        Auto-generated to hold the location of the Android SDK on the current machine.
jni/Android.mk          sub-makefile that describes the C++ static libraries and *.so files you want to build
jni/Application.mk      contains settings that apply to all the C/C++ code 
                        (processor type, API level, standard library, project-wide compiler options)
build.xml               Unknown (required by vs-android only)
                        Note: in vs-android samples, the .* files are in the parent folder of 
                        everything else. But personally I have reconfigured my project so that these files 
                        are in the same folder as the files above.
(projname).vcxproj      vs-android project file
(projname).vcxproj.filters vs-android secondary project file****
(projname).sln          Visual studio solution file
(projname).suo          Auto-generated by Visual Studio for local user settings
(projname).vcxproj.user Auto-generated by Visual Studio for local user settings (one file is not enough?!)
(projname).sdf          Auto-generated by vs-android or VS. Probably a C++ symbol database (SQL Server Compact)
* .project is the Eclipse analogue of a Visual Studio project file, sort of, but it doesn't contain a list of the files that are in the project; in Eclipse, everything in the directory tree is implicitly part of the project. The project name is stored in the .project file, but as far as I can tell, this name is only cosmetic; it is not used in the output *.apk.
** default.properties or project.properties merely repeats the target SDK version from AndroidManifest.xml. A comment in the file claims the file is automatically generated, but apparently it is generated only once (on project creation). Eclipse will barf with a series of NullReferenceExceptions if the file is missing. For some reason there are two possible names for it; some projects use one name, others use the other name.
*** ProGuard is an obfuscation tool. I didn't ask for obfuscation, but Eclipse created proguard.cfg anyway.
**** *.vcxproj.filters specifies how files will be arranged into "Filters" (tree nodes that look like folders) in Visual Studio. I can't imagine why this information is not simply stored in the *.vcxproj file.

Android + NDK tips, part 1: Glossary

Android Glossary - the minimum you need to know
  • Activity: One screen of an application. Similar to a top-level window on other platforms. 
  • View: Established terminology be damned. A "control" or "widget" on all other platforms is a "view" on Android. 
  • Layout XML: Describes an arrangement of views. Equivalent to a dialog resource on Windows, but in XML. Android layouts are auto-resizing, like WPF/XAML, but don't support data binding and must be accessed in a tedious manner via resource ID numbers in Java, much like MFC/Win32.
  • Layout class a.k.a. ViewGroup: A container for a set of Views, that is, a parent control. Most ViewGroups control the way their children are arranged, except for the AbsoluteLayout. Probably the most common ViewGroup is LinearLayout, which stacks controls vertically or horizontally (but not both).
  • Intent: A Java class that represents an action to perform. The most common Intent is a request to start an Activity.
  • Eclipse: The world's most popular open-source IDE (and perhaps the most resource-hungry).
  • Dalvik: The thing that runs Java code on Android.
  • R: An auto-generated java class that holds all the application's resource IDs.
  • NDK: Native (C/C++) development kit.
  • JNI: Java native interface. A system whereby Java can call C functions and C/C++ can call Java through a reflection-like interface. Unlike C# P/Invoke, Java JNI cannot call arbitrary C functions; the C functions must be specifically designed for JNI.
  • lib*.so: Shared object. Equivalent to *.dll on Windows, but *.so files MUST start with "lib".
  • *.apk: Android package. Comparable to a *.cab on Windows Mobile or a Java *.jar file, the *.apk is actually a *.zip file in disguise. It holds your application, all of its resources and bitmaps, and any *.so files.

Wednesday, May 02, 2012

Fun with income tax forms

Somewhere in Alberta, Canada, I fill out my 2011 tax forms...

Even though I'm in the 22% Federal income tax bracket, and the provincial tax rate is 10%, it looks like I'll be paying only about 6% of my total income in taxes this year (plus about 5% for CPP/EI). How the heck is that possible, you ask? It's a shame that the tax forms are so hard to follow, because it's quite useful to know how they work.
  • [T1 Line 208] Well, I bought over $10,000 in RRSPs and those are deducted from my income right away, taking me near the bottom of the 22% tax bracket (and closer to the 15% bracket, which is the lowest tax bracket).
  • [Schedule 1 Line 1,7,9,12] Next, like everyone else I get to deduct the basic personal amount of $10,527, CPP and EI premiums (about $3000 for both) and an extra $1065 called the "Canada employment amount" which I apparently get to deduct simply because I have a job. Unlike the RRSPs which cut off the "top" of your income, these other deductions cut off the "bottom" of your income since they only count at 15%, not at 22% or whatever tax bracket you're in.
  • Notice that the basic personal amount is an odd beast: the government could have simply created a 0% tax bracket for the first $10,527, but that's not technically how it works. The lowest tax bracket is 15%, but the first  $10,527 is counted as a non-refundable tax credit. Since this number is added in with all the other tax credits, it makes your tax forms more difficult to follow, but as far as I can see, the end result is exactly the same as if a 0% tax bracket did exist (even if you are in one of the higher tax brackets).
  • [Schedule 1 Line 51] Federal political contributions of $400 or less are deducted at an incredible 75% rate! So if you're in the 15% bracket, it is as though the government does not tax your contribution at all and then pays 60% of the money back to you! I took advantage of this weird little-known rule to give the Pirate Party $400, which, depending on how you look at it, cost as little as $100. (You can give over $400, of course; contributions between $400 and $750 are deducted at 50%, and up to $1275 are deducted at 33%.)
  • [Schedule 1 Line 30 + Schedule 9] The real gravy train is in charitable donations. When you give more than $200 to charity, the federal government deducts everything over $200 at a rate of 29% (equivalent to the highest tax bracket)*. If you are squarely in the 22% bracket, that's as if the federal government is not merely deducting the donations, but actually paying you 7% of what you donate back to you. But in my case, being near the bottom of the 22% bracket, it is more like the FG is paying me about 10% back. (* the bottom $200 is deducted at only 15%.) I wonder how much tax the charities have to pay on donations...
  • But that's not all! You also have to pay provincial tax! But provincial taxes are much lower than federal taxes, at least here in Alberta. Alberta seems to have a flat tax of 10%, so the forms don't hurt your head as much. Albertans get to deduct a basic personal amount of $16,977 plus CPP and EI premiums (about $3000 for mine, just as before, for total deductions of about $20,000 at 10%). Note that RRSPs are deducted "in advance", since the provincial form calculates starting from the taxable income (total income minus RRSPs) instead of the total income.
  • [AB428 Line 27] So as I was saying about the gravy train... in Alberta your charitable donations above $200 are deducted at 21%, 11% above the 10% tax rate. So not only are your donations tax-deductable, but the provincial government pays 11% of your donation back to you!
  • To recap, in Alberta, charitable donations above $200 are deducted at a rate of 50%, independent of your income (21% provincial + 29% federal) even though the total tax rate is only 25% to 39% depending on your income. I donated $7400, so the total deduction is almost $3700 (not quite, since the deduction is only 25% on the first $200.)

    Great deal, huh? The only problem is, only registered Canadian charities get to take part in the gravy train. Some of my favorite charities such as the EFF and Avaaz are not registered Canadian charities (Avaaz says it cannot be a registered charity because it is politically active; EFF is not a registered Canadian charity because it's a US charity). Just don't go overboard with donations, because they are non-refundable: you can't take your tax rate below 0%.
Tip for anyone using StudioTax: yes, the UI is very confusing. If you can't find one of the forms on the list of forms on the left side, click the "Forms" button on the toolbar. This will open a dialog that contains a list of dozens of hidden forms. Find the one you want to see from the top area, select it, then click the button that has a downward-arrow on it, and finally click OK.