My build system currently contains 14 Java EE web applications and 49 Java library projects. Back when it was only 6 projects and one web application, I performed builds and dependency management manually. About a dozen projects was the tipping point for me where I subsequently added Ant for builds, Ivy for dependency resolution, and finally closed the loop about 5 years ago by moving builds entirely to Jenkins and Artifactory. Our core projects have 100% branch coverage, so QA/QC is essentially guaranteed via the build process. This saved me so much time; it is like having a whole team handling my builds and testing for me. I remember the bad old days where I would spend 30 minutes coding and 2 hours building and deploying. Now Jenkins handles the full build cascade, which on a single server instance takes right around an hour if the core projects need to be built and tested. Most of the time however, I’m just committing changes to the web applications, which take only a minute or so to build and deploy.

Here is my current process:

1. Commit code to private SubVersion repository.
2. SubVersion automatically triggers Jenkins build when commit completes.
3. Jenkins build (fully automated).
   • Jenkins updates its local repository.
   • Jenkins runs Ant build.
     • Ant resolves Ivy dependencies via private Artifactory instance.
     • Ant performs build and javadoc.
     • Ant runs unit tests (JUnit) with code coverage reports (Cobertura).
   • Jenkins publishes artifacts to private Artifactory instance if build was successful.
   • Jenkins deploys web application(s) to development server (Tomcat) if build was successful.
   • Jenkins triggers step 3 for any downstream projects.
4. Local development environment (Eclipse with IvyDE) resolves the newly deployed artifacts from private Artifactory instance.
5. Deployments of web applications to production server occur at most once per week and are handled manually via Tomcat manager. This is trivial but I want a better way to manage deployments to production.

I’m attempting to document the installation of Artifactory 6.2.0 on an Amazon Linux instance. I have a working instance of Artifactory 5.3.2, but in a minor version update (5.3.3) they split their authentication system into a separate web application, and since then I’ve made several unsuccessful attempts at upgrading. I would dig into the source code, except it isn’t currently available. Time to decompile, I guess. Sometimes I get the impression that JFrog isn’t using their own software effectively.

Update: Got it. It turns out that the Artifactory Access Server/Client communicate over the Server HTTP response parameter, which is overwritten by Apache web server. Details are here and I will probably edit that into a full tutorial.

I am taking the rest of the weekend off.

I just finished developing a workaround for a Spring Security bug. It turns out, a new HttpFirewall feature enabled by default in Spring Security 4.2.4 (April 2018) was silently dropping all requests from clients that did not support cookies. The Spring Security team has a fix slated for version 5.1 (time undetermined). My workaround puts web application developers back in control of the firewall feature.

I implemented a workaround for a Spring Security issue that has been driving some people crazy since April. Basically, a new firewall feature was rolled out in a minor version update (v4.2.4) but provided no way for the developer to control the logging or blocking behavior. This workaround gives the developer a lot more control over this feature.

The link to the actual paper is here. For the record, I hope that this idea never makes it into common practice. It seems like a return to security through obscurity, with the added detriment of increasing development, debugging and testing complexity.

Alternatively, here is an equivalent XSL stylesheet that can be applied directly to a sitemap XML document:

<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet xmlns:sitemap="http://www.sitemaps.org/schemas/sitemap/0.9" xmlns:xsl="http://www.w3.org/1999/XSL/Transform" xmlns:xhtml="http://www.w3.org/1999/xhtml">
  <xsl:output encoding="UTF-8" method="text" version="1.0" />
  <xsl:template match="/"><xsl:apply-templates select="sitemap:urlset/sitemap:url/sitemap:loc" /></xsl:template>
  <xsl:template match="sitemap:loc"><xsl:value-of select="text()" /><xsl:text>
</xsl:text></xsl:template>
</xsl:stylesheet>

This has the added benefit of not requiring the whitespace-processing step in the article, and could be easily modified to select elements based on attributes (e.g., language). However, it does require that the sitemap document actually be well-formed XML – the example in the article is not well-formed because it contains an unclosed <urlset> element. This is why XML parsers, validating parsers and transformers are so much more valuable than ad-hoc implementations. If the document does not conform to the specification, it will tell you.

I found it a little hard to understand the aims and tradeoffs of this algorithm from wikipedia. The original paper is here: https://cr.yp.to/bib/1975/aho.pdf

This algorithm lets you search for all occurrences of many short patterns in a long text.

One application might be syntax hilighting keywords in a text editor.

It works by building a finite state machine out of a list of patterns. This is similar to the common lexing technique we use of making a vector of regular expressions and running them all “in parallel”.

RSS was a great concept (and appropriate for its time), but was designed by people who didn’t comprehend XML namespaces, instead forcing implementations (both generators and readers) to escape XML and/or HTML tags, which requires multiple passes for generating and parsing feeds - with an intermediate encoding/decoding step (Really Simple?). They purportedly addressed this in RSS 2.0, but if you have a look at their RSS 2.0 example, they still got it wrong, persisting a 1990’s understanding of the web. Although I still use it, I shake my head in disappointment every time I see RSS source. RSS 2.0 should really have been based on something that could be validated, such as XHTML.

At this point, it is probably way too late for a comeback, as:

1. Social media platforms like Twitter are commonly used as a substitute and have a large hegemony over content.
2. Browsers have given up on RSS in favor of their own peculiar readers.
3. Google, Microsoft, Yandex and whatever Yahoo is now are pushing for an entirely different system based on extracting information from HTML content via an ever-changing pseudo-ontology that lacks definitions and is inconsistently employed by every practitioner.

You could read the above points as things that RSS should be able to overcome. If RSS were indeed to make a comeback, I would hope that in a new “RSS 3.0” incarnation it would satisfy the following criteria:

1. Standard comes before implementation (e.g., utilize existing standards).
2. Validatable (e.g., employ XML namespaces and utilize an XSD for document validation).
3. Human readable (i.e., subset of XHTML or HTML, that can be consistently rendered as in any modern web browser)
4. Strict specification (use a well-defined structure with a minimal tag set that prevents multiple interpretations of the specification).

I’ll admit, I do not like JSON one bit because it is antithetical to several, if not all of the above criteria. However, since a JSON alternative is desired, I would recommend that it be directly based on an XML/HTML version that does satisfy the above criteria. Then a simple XSL (read “standardized”) spreadsheet could be employed to generate the equivalent JSON version, satisfying both worlds.

I’m not following the conspiracy theory with Hollywood. What evidence is available for that assertion?

Vulnerability patches for Intel ME are a half-measure and not forward-thinking. Siding with the EFF position, I would like to see Intel officially endorse a ME removal method. To put this problem in perspective, Intel embedded a secret, MINIX-based OS (complete with web server) into their Core-based CPUs starting in 2008, and only conducted a security review of it in 2017 after third-party researchers identified it as highly exploitable in 2016. Intel had already provided the NSA with a method for disabling ME, so it is clear that the risks were well known for years prior to public disclosure. Intel provides a Risk Assessment Tool for some vulnerabilities but it is not clear if they will continue to update it as new vulnerabilities are disclosed.

What a delightful read.

In a somewhat related direction, I am absolutely convinced that the neophilia and constant complication of otherwise straightforward projects is a neurotic response similar to parrots plucking feathers and mutilating themselves when given not enough to do.

I’ll write about this at length later, but the short summary of it is that developers realize that they’re given often basic tasks and rather than embrace the fact that those problems are always well-solved and could be embraced with just a little bit of pushback on the people side of things and maybe encouraging just buying a solution off-the-shelf or upgrading to better hardware to run that long ETL, they instead fixate on new “best practices” and ignore the creeping existential horrors (mokitas!) that:

• most practical interesting problems in CS are solved well, or can be solved by throwing money/hardware at them
• businesses make money by making people feel like their problems are solved–not always by solving those problems or even creating anything interesting
• developers are at once both overpaid for the drudgery and simple tasks that they do, and underpaid and cut out of the growth of wealth they enable
• increasingly the narrative pushed to others is that devs are the source of their problems and they should be shunned
• employers don’t actually give a shit about growing their developers
• developers are locked-down from owning the fruits of their intellect

And on and on and on. So, of course, we see this neurotic behavior in our industry, and dress it up as something else instead.

These are by no means advanced techniques. The blog post cherry-picks a subset of suggestions from this 2013 post, some of which aren’t even specific to Java garbage collection:

1. provide collections with expected size upon construction
2. don’t load large files into memory
3. use immutable objects where possible.

A quick google search will find more shorter, yet more in-depth articles on this topic: Reduce Long GC Pauses. Regardless of general strategies, the best approach to improving performance and avoiding GC pauses is to attach a profiler like Visual VM to see how many objects are being created by each method and how much memory they are using.

von Neumann would roll over in his grave if he heard people complaining about contemporary programming languages. What was the point of this article? The author is either doing some high-level trolling, has some serious projection issues or is proudly showing off both incompetence and an unwillingness to learn. There isn’t a single coherent thread through the 1,600 words of text – the journalistic equivalent of spaghetti code. If he writes software in a similar fashion, of course he will get frustrated. I mostly write code in Java, and I do not encounter any of the issues or frustration he does. But, I do not expect a compiler or runtime interpreter to understand the general idea of what I’m trying to accomplish, nor would I want to write software to perform a task that I can’t coherently explain to a co-worker.

Recovering from a half-marathon. Testing power consumption for various hardware configurations for the Linux intel_idle patch I wrote last weekend. At idle in power state C4E, I shouldn’t be hearing the CPU fan. And yet there is still fan noise.

I’ve been waiting for an in-depth discussion of this issue, and this covers all the bases that are relevant to my company. Fortunately, when I migrated our production server from self-hosting to Amazon Linux in 2013, we started using the free OpenJDK since it was the default on that distribution. This was forward thinking on Amazon’s part (as well as Sun Microsystems, RIP), and in retrospect, much appreciated. We had zero compatibility issues, we are always on the latest version, and we owe zero royalties.

I arrived at a similar conclusion a few years ago when I encountered differences between what the language specification and the compiler considered to be correct type-casting. Fortunately, generics in Java are essentially syntactic hints to the compiler and do not end up in the compiled bytecode (this both enables and prohibits some useful but possibly unintended features involving multiple inheritance over interfaces).

I started out with Slackware 2.1 in 1994, and it is still my go-to Linux distribution for self-hosted servers. The installation and configuration has changed very little, and when combined with SlackBuilds, it is as current and competitive as other distribution out there. I’m currently finalizing the deployment of a Slackware-current (14.2+) GPU+CPU compute cluster for genome comparison. NOAA Gulf Coast hurricane forecast models also run on a Slackware cluster.

I am adding Intel Core 2 support to the Linux intel_idle driver so that Q45 Express chipsets with Yorkfield, Wolfdale, Conroe, Penryn or Merom processors can take advantage of extended/deeper power management states. Would have been nice to have 8 years ago, but better late than never.