Hi there! I'm so glad you've found my blog! My blog is now: codeblast.com
Over the years, I've just never been able to find a blogging platform that I'm happy with. Blogger has been really good from an admin perspective, but not so much for WRITING blog posts. I prefer Markdown as a file format, and GitHub Pages supports [Markdown](https://pages.github.com/) very well.
I still haven't found a blogging workflow that motivates me to blog more, but for what it's worth I've decided to switch to GitHub Pages for now.
I may perhaps try to port my old blogposts from this Blogger site (and maybe even older ones from WordPress) to my new blog some day. But in the future, please look for me at codeblast.com.
Also please consider joining Twitter and follow me at @codeblast. Hope to hear from you!
Update 2/28/2016:
Today I finally imported my old posts from my previous Blogger website at www.lombard.me to GitHub Pages using the jekyll-import Ruby gem... Unfortunately I couldn't import the comments from Blogger, so they'll be lost. I'm now using [Disqus](https://disqus.com/) for the comments system.
When you’re using Dagger 2.0 in your Android projects, you’ll quickly realize you need to include
an additional dependency to provide the @Generated annotation (from JSR-250). [**]
These are the common options typically seen in online Dagger examples:
So the second contains slightly more meta-data files and a different license (includes GPL) than the first option, but otherwise they contain essentially the same content - e.g. the same .class files.
The third one contains a few extra annotations, including: @ManagedBean, @Priority, but is pretty much the same as the second one.
Verdict: use the first/oldest one: javax.annotation:jsr250-api:1.0
Final note: when you include the dependency in your android project, use the provided scope instead of compile, to prevent the JSR-250 annotations from being distributed with your app, in other words:
** PS. if you don’t include one of these JSR-250 jars in your project, you’ll get this compile error in one of the files generated by dagger-compiler:
Error:(17, 24) error: package javax.annotation does not exist
Error:(20, 2) error: cannot find symbol class Generated
Error:(21, 14) error: dagger.internal.codegen.ComponentProcessor was unable to process this class because not all of its dependencies could be resolved. Check for compilation errors or a circular dependency with generated code.
Your CI server doesn’t have direct internet access; or
it requires a proxy for internet access
and your Robolectric test builds are failing as result…
The problem
The Robolectric test runner downloads some dependency jars that it needs to run at runtime (not compile-time). Normally you don’t even notice this when running your unit tests locally, but this could be a problem on Jenkins/CI if your company doesn’t allow internet access from the CI boxes (as it should be). What happens is, when the Robolectric tests try to run on your CI box, it will fail with a MultipleArtifactsNotFoundException error like the following:
If you run gradlew with -i or -d options, you’ll see more details of what’s happening and then you realize it’s because Robolectric is trying to download jars from repo1.maven.org which you may not have access to if you’re using a local repo such as Artifactory.
We want to stop Robolectric from trying to download its jars from Maven Central and force it to use a local copy of the jars, which we can manually copy to the CI build slave servers.
Robolectric 2.4 introduced two new settings to prevent downloading the dependencies from the hardcoded sonatype / maven URL:
robolectric.offline
robolectric.dependency.dir
However, it’s not completely clear from the documentation how to use these settings. The documentation just mentions that they’re “system properties”.
There were a few questions I had when I started with this:
Which jars are needed by the Robolectric test runner?
How do I configure these two system properties in my project’s build.gradle file?
The solution
Step 1: Which jars are needed?
In order to place the jars in a known location on the CI build slaves, we first need to figure out exactly which jars are used by Robolectric during run-time.
One way to do this, is to run gradlew testDebug -d (i.e. with debug output) and examine the logs to figure out which jars were downloaded during test run-time.
I decided to take another approach, and go to the Robolectric sources to see which dependencies it needs. The info we want is in SdkConfig.java.
From SdkConfig.java I could tell that the following four dependencies are needed by Robolectric 3.0 at run-time:
However, we also need the transitive dependencies of these jars, i.e. the dependencies of these dependencies.
I figured the easiest way to correctly download these 4 jars and all their dependencies, is to let Maven do it. We can create a simple Maven pom.xml file that just lists these 4 dependency artifacts, and then we let mvn dependency:copy-dependencies download everything for us.
$ mvn dependency:copy-dependencies -DremoteRepositories=http://repo1.maven.org/maven2/ -DoutputDirectory=/tmp/robolectric-files
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building robolectric-files 3.0-rc2
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-dependency-plugin:2.8:copy-dependencies (default-cli) @ robolectric-files ---
[INFO] Copying vtd-xml-2.11.jar to /tmp/robolectric-files/vtd-xml-2.11.jar
[INFO] Copying accessibility-test-framework-1.0.jar to /tmp/robolectric-files/accessibility-test-framework-1.0.jar
[INFO] Copying tagsoup-1.2.jar to /tmp/robolectric-files/tagsoup-1.2.jar
[INFO] Copying json-20080701.jar to /tmp/robolectric-files/json-20080701.jar
[INFO] Copying robolectric-resources-3.0-rc2.jar to /tmp/robolectric-files/robolectric-resources-3.0-rc2.jar
[INFO] Copying shadows-core-3.0-rc2-21.jar to /tmp/robolectric-files/shadows-core-3.0-rc2-21.jar
[INFO] Copying sqlite4java-0.282.jar to /tmp/robolectric-files/sqlite4java-0.282.jar
[INFO] Copying icu4j-53.1.jar to /tmp/robolectric-files/icu4j-53.1.jar
[INFO] Copying android-all-5.0.0_r2-robolectric-1.jar to /tmp/robolectric-files/android-all-5.0.0_r2-robolectric-1.jar
[INFO] Copying robolectric-annotations-3.0-rc2.jar to /tmp/robolectric-files/robolectric-annotations-3.0-rc2.jar
[INFO] Copying robolectric-utils-3.0-rc2.jar to /tmp/robolectric-files/robolectric-utils-3.0-rc2.jar
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 08:55 min
[INFO] Finished at: 2015-05-13T18:26:23-07:00
[INFO] Final Memory: 13M/239M
[INFO] ------------------------------------------------------------------------
Now copy all those files from /tmp/robolectric-files/ to some known location on your CI build slave box, say to /home/jenkins/robolectric-files (assuming jenkins is your CI build user account).
Step 2: Set the offline-mode System Properties in build.gradle
We need to set the following two Java System Properties for the Robolectric Test Runner to use the local files:
Now we can run Gradle and our unit tests completely in offline mode:
$ ./gradlew clean testDebug --offline
(Note: hardcoding a specific user’s home directory in the gradle file isn’t a good idea, but you get the point. Use a directory that works for you.)
Alternate solution:
Instead of Step 2 above, of course you could just install the necessary files into your CI box’s Maven repository (~/.m2/repository/) using something like this for each of them:
But in my solution above I opted for overriding the location of the files using the robolectric.offline and robolectric.dependency.dir system settings, because that way I have more clear control and visibility over which files I’m responsible for maintaining manually.
Update 17 July 2015:
My python script above worked for robolectric-3.0-rc2 but not the final 3.0 version pom.xml… See @benoberkfel’s comment.
Create a file called ~/.m2/settings.xml and specify a mirror URL for your local Artifactory/Nexus repository. That way Robolectric will automatically download the required runtime files from your local mirror and no other hacks are required. This seems like a nicer solution depending on your needs. Something like this:
If you’re a developer or DevOps-type guy, then I bet you like doing things on the command-line and/or in batch files / shell scripts.
This year I’ve discovered two amazing tools for querying XML and JSON documents from the command-line: XMLStarlet for querying XML and jq for querying JSON from the command-line. Both tools are available on Mac OS X, Windows and Linux. Right now I want to focus on XMLStarlet.
XMLStarlet lets you make XPath style queries and transformations right from the command-line.
This is very handy to query Web APIs or local XML files from scripts.
For example, I often want to parse the results out of checkstyle, pmd, findbugs or junit XML report files in an Android project.
In this post I’ll show how to use xmlstarlet to query the checkstyle XML report output for an Android project.
On Windows, Chocolatey will probably work for you:
choco install xmlstarlet
XMLStarlet select basics:
Do this first: look at the sel command’s help (i.e. select data using XPath):
$ xmlstarlet sel --help
Select command help output:
XMLStarlet Toolkit: Select from XML document(s)
Usage: xmlstarlet sel <global-options> {<template>} [ <xml-file> ... ]
where
<global-options> - global options for selecting
<xml-file> - input XML document file name/uri (stdin is used if missing)
<template> - template for querying XML document with following syntax:
<global-options> are:
-Q or --quiet - do not write anything to standard output.
-C or --comp - display generated XSLT
-R or --root - print root element <xsl-select>
-T or --text - output is text (default is XML)
-I or --indent - indent output
-D or --xml-decl - do not omit xml declaration line
-B or --noblanks - remove insignificant spaces from XML tree
-E or --encode <encoding> - output in the given encoding (utf-8, unicode...)
-N <name>=<value> - predefine namespaces (name without 'xmlns:')
ex: xsql=urn:oracle-xsql
Multiple -N options are allowed.
--net - allow fetch DTDs or entities over network
--help - display help
Syntax for templates: -t|--template <options>
where <options>
-c or --copy-of <xpath> - print copy of XPATH expression
-v or --value-of <xpath> - print value of XPATH expression
-o or --output <string> - output string literal
-n or --nl - print new line
-f or --inp-name - print input file name (or URL)
-m or --match <xpath> - match XPATH expression
--var <name><value> --break or
--var <name>=<value> - declare a variable (referenced by $name)
-i or --if <test-xpath> - check condition <xsl:iftest="test-xpath">
--elif <test-xpath> - check condition if previous conditions failed
--else - check if previous conditions failed
-e or --elem <name> - print out element <xsl:elementname="name">
-a or --attr <name> - add attribute <xsl:attributename="name">
-b or --break - break nesting
-s or --sort op xpath - sort in order (used after -m) where
op is X:Y:Z,
X is A - for order="ascending"
X is D - for order="descending"
Y is N - for data-type="numeric"
Y is T - for data-type="text"
Z is U - for case-order="upper-first"
Z is L - for case-order="lower-first"
(rest of output omitted for brevity...)
The most important ones to note are the -v (value-of) and -i (if) expressions. Also useful is -o (literal string output) and -n (print newline in the output).
Parsing checkstyle.xml report to list errors:
Let’s break things down. First you need to find all the checkstyle.xml report files for your project. This will depend on your Gradle setup, but for me this works (from the project root):
This lists all the checkstyle.xml reports under each Android module in my project.
The contents of the checkstyle.xml file has this syntax:
<?xml version="1.0" encoding="UTF-8"?><checkstyleversion="6.2"><filename="/Users/glombard/myproject/app/src/main/java/com/codeblast/android/myapp/MyFile.java"></file><filename="/Users/glombard/myproject/app/src/main/java/com/codeblast/android/myapp/OtherFile.java"><errorline="7"severity="error"message="Wrong order for 'android.animation.ValueAnimator' import. Order should be: android, com, junit, net, org, java, javax. Each group should be separated by a single blank line."source="com.puppycrawl.tools.checkstyle.checks.imports.ImportOrderCheck"/></file>
... rest omitted...
This means that MyFile.java has no errors, but the second entry, OtherFile.java has one error with the order of the imports at line 7.
I’d like to parse out the errors from this file.
To make it easier, let’s first just list all the files checked, regardless of whether they’ve got errors or not:
$ xmlstarlet sel -t -m /checkstyle/file -v @name -n library/build/reports/checkstyle/checkstyle.xml
OK, that gives us a taste for the -m (match element) and -v (print value-of) commands (we’re just printing the name attribute of the <file> element…)
But we’re actually only interested in files containing <error> child elements. Let’s only list files with errors by first matching on /checkstyle/file and then also matching on error:
This just prints out the filenames of the file elements that also had error elements in them.
Now let’s put everything together: we’ll use find to find all the checkstyle.xml files in our project, then we want to use xargs to pipe the contents of all those checkstyle.xml files to xmlstarlet and then query for the errors:
I hope this helps you to see the possibilities of being able to query XML from scripts (CI etc). If so, let me know how. If not, let me know how I can improve this page. :-)
Remember, also check out jq for querying JSON files in a similar way.
Ever since I created my first website(s) on Xoom and Tripod around 1998, before the days of blogging, I’ve wanted to blog. I have these random thoughts about things and I feel the desire to write it down. More for myself than for any particular audience. Just because I’m attracted by the idea of creating something.
However, I’ve been trying to write blog posts occasionally over the last few years, and I keep getting discouraged by my lack of writing skill. I start typing a few notes for a blog idea, and then once I get to a few bullet points of ideas, I really struggle to figure out how I’m going to expand the handful of bullet points into a full blog post with colors and graphs and code samples etc. Hundreds of potential blog posts only remained as small draft note files somewhere in a “blog ideas” directory.
I look at blogs by some of my blogging heros like Scott Hanselman, Jon Sonmez, Erik Dietrich or Claire Burge (just a few random examples), and I feel both inspired and discouraged at the same time by the quality and quantity of their blogs (and other content!). I know I’ll never have the colorful vocabulary and the skill with words to create something that I could be that proud of. (And I’ve never been very inspired to blog using the tools I’ve tried like WordPress, Blogger, LiveWriter, but I’m hoping that will change now I’m using GitHub Pages.)
I have to keep reminding myself: screw it, let’s do it. The guilt I feel about not blogging at all (or not writing more code for that matter) is worse than my disappointment in my uninspiring writing.
I think a lot of developers don’t find blogging appealing because they think they’ll leave the blogging to the experts and I guess they don’t feel comfortable putting their opinions out there for everyone to see. I don’t really care much about what people think about my blog posts. If a stupid blog post I wrote 2 years ago helps one person to configure their internet settings on their Nexus 7 for O2 Ireland, then I think that’s pretty cool. But ultimately I’m really just doing it for myself. I’m talking to myself, because it allows me to try and structure my thoughts in a way I can’t really do if I keep the thought in my brain. Ideally I would have discussed the ideas instead (or as well) with a friend or a coworker to have a more interesting two-way conversation, but this will have to do.
So thanks for being a listening friend here.
PS. I also wrote about this same topic back in 2012.