Saturday, February 14, 2015

Sonatype Nexus: Retrieving artifacts using the REST API or Apache Ivy

Sonatype Nexus is an often used artifact repository. In a previous blog post I have shown an example how Maven can be used to assemble and release artifacts to Nexus. In this blog post I will describe two ways how artifacts can be fetched from the Nexus repository; by using the REST API and by using Apache Ivy.


The Nexus API

Sonatype has done a great job at providing an extensive well documented REST API. See: http://www.sonatype.org/nexus/2015/01/26/learn-the-nexus-rest-api-automating-sonatype-nexus/. The webinterface which allows administration and browsing of the repository, makes use of this API and is thus provides many examples on how the API can be used. You can use a network traffic monitor (Chrome Developer tools for instance) to obtain the requests send from the webinterface to the API.

You can also try requests on the API by using for example SOAP UI or an internet browser. SOAP UI is in my opinion the more user friendly way to experiment.


Below are some examples on how you can use this API. I have used the same Nexus installation and testproject as in the previous mentioned post.

The keyword LATEST is used in the samples. This only works (as far as I've seen) if a pom file is present in the artifact directory (the Maven resolver can be used).

POM

In order to fetch the latest version of a project, you can perform a query like below. This fetches the POM (Maven project object model).

http://localhost:8081/nexus/service/local/artifact/maven?g=nl.amis.smeetsm.application&a=testproject&v=LATEST&r=releases

The response XML conforms to the following XSD: https://repository.sonatype.org/nexus-restlet1x-plugin/default/docs/ns0.xsd.(which contains a duplicate definition...)

Browsing

You can browse the directory structure. Suppose all your services have a specific groupid, you can query artifacts from the group and use some scripting to gather the required artifact. For example;

http://localhost:8081/nexus/service/local/repositories/releases/index_content/nl/amis/smeetsm/application/testproject/

Gives me an XML describing the artifacts and versions of my testproject. You can use this response in for example a simple Python script to query for artifacts from outside the webinterface. When applying an XSL, you can also easily transform the XML to a nice viewable HTML.

Fetching an artifact directly

You can fetch an artifact directly by going to an URL like for example below;

http://localhost:8081/nexus/service/local/repositories/releases/content/nl/amis/smeetsm/application/testproject/1.2/testproject-1.2-distribution.zip

Query for an artifact

Or you can use a query based on the GAV (group id, artifact id, version) coordinates and the filename to fetch the latest version.

http://localhost:8081/nexus/service/local/artifact/maven/content?g=nl.amis.smeetsm.application&a=testproject&v=LATEST&r=releases&c=distribution&e=zip

Apache Ivy

Apache Ivy can be used in Ant tasks to make dependency management easier. Apache Ivy easily integrates with Maven repositories (like Nexus) while still allowing usage of the extensive scripting options of Ant. The Nexus API allows resolving the latest version of Maven artifacts. When using Apache Ivy however to publish artifacts and you do not have a separate pom.xml file, you are dependent on the Apache Ivy resolver for obtaining the latest version of artifacts. Apache Ivy however provides Ant tasks for generating a pom file if you do not already have one (makepom).

What do you need to get Apache Ivy fetching artifacts from Nexus for you? There are several examples online, but I could not find a complete one so I'll provide it here. First download Apache Ivy and put the jar file in a lib directory

ivy.xml

You can compare this with a Maven pom.xml file. It describes an artifact.

 <ivy-module version="2.0" xmlns:maven="http://maven.apache.org">  
   <info organisation="nl.amis.smeetsm.application" module="ivytest"/>  
   <configurations>  
         <conf name="runtime" description="runtime" />  
   </configurations>  
   <dependencies>  
     <dependency org="nl.amis.smeetsm.application" name="testproject" rev="1.1" conf="runtime->default">  
         <artifact name="testproject" maven:classifier="distribution" type="zip" ext="zip"/>  
   </dependency>  
   </dependencies>  
 </ivy-module>  

ivysettings.xml

Describes stuff like credentials and resolvers. You can compare this with a Maven settings.xml

 <ivysettings>  
  <settings defaultResolver="nexus"/>  
  <credentials host="localhost" realm="Sonatype Nexus Repository Manager" username="deployment" passwd="deployment123"/>  
  <property name="nexus-public" value="http://localhost:8081/nexus/content/groups/public"/>  
  <resolvers>  
   <ibiblio name="nexus" m2compatible="true" root="${nexus-public}"/>  
  </resolvers>  
 </ivysettings>  

build.xml

This is an Ant script which can be executed by calling the Ant binary in the directory which contains this file. Important here is that the combination of the artifact definition in ivy.xml and the pattern in the ivy:retrieve call, determine which file is actually fetched.

 <project name="ivytestproject" default="init"  
 xmlns:ivy="antlib:org.apache.  
 ivy.ant">  
      <!--  
   ================  
   Build properties  
   ================  
   -->  
      <property name="build.dir" location="build"/>  
      <property name="ivy.reports.dir" location="${build.dir}/ivy-reports"/>  
      <!--  
   ===========  
   Build setup  
   ===========  
   -->  
      <target name="init">  
           <ivy:settings file="ivysettings.xml" />  
           <ivy:retrieve pattern="${build.dir}/[artifact](-[revision])(-[classifier]).[ext]"/>  
           <ivy:report todir='${ivy.reports.dir}' graph='false' xml='false'/>  
           <ivy:cachepath pathid="runtime.path" conf="runtime"/>  
      </target>  
 </project>  

Result

When executing Ant in the same directory as the build.xml, Ivy goes fetch the artifact from Nexus and puts it in a build directory. There you can do whatever Ant scripting you like with it. In the ant commandline, you need to specify the library directory in which you have put Apache Ivy (and probably other Ant dependencies). See the example below.

 [maarten@hotspot ivytest]$ cat runme.sh  
 #!/bin/sh  
 /home/maarten/Oracle/Middleware1213/Oracle_Home/oracle_common/modules/org.apache.ant_1.9.2/bin/ant -lib /home/maarten/ivytest/apache-ivy-2.4.0-rc1  
 [maarten@hotspot ivytest]$ ./runme.sh  
 Buildfile: /home/maarten/ivytest/build.xml  
 init:  
 [ivy:retrieve] :: Apache Ivy 2.4.0-rc1 - 20140315220245 :: http://ant.apache.org/ivy/ ::  
 [ivy:retrieve] :: loading settings :: file = /home/maarten/ivytest/ivysettings.xml  
 [ivy:retrieve] :: resolving dependencies :: nl.amis.smeetsm.application#ivytest;working@hotspot.s-bit.nl  
 [ivy:retrieve]   confs: [runtime]  
 [ivy:retrieve]   found nl.amis.smeetsm.application#testproject;1.1 in nexus  
 [ivy:retrieve] :: resolution report :: resolve 85ms :: artifacts dl 4ms  
   ---------------------------------------------------------------------  
   |         |      modules      ||  artifacts  |  
   |    conf    | number| search|dwnlded|evicted|| number|dwnlded|  
   ---------------------------------------------------------------------  
   |   runtime   |  1  |  0  |  0  |  0  ||  1  |  0  |  
   ---------------------------------------------------------------------  
 [ivy:retrieve] :: retrieving :: nl.amis.smeetsm.application#ivytest  
 [ivy:retrieve]   confs: [runtime]  
 [ivy:retrieve]   1 artifacts copied, 0 already retrieved (0kB/7ms)  
 [ivy:report] Processing /home/maarten/.ivy2/cache/nl.amis.smeetsm.application-ivytest-runtime.xml to /home/maarten/ivytest/build/ivy-reports/nl.amis.smeetsm.application-ivytest-runtime.html  
 BUILD SUCCESSFUL  
 Total time: 6 seconds  
 [maarten@hotspot ivytest]$ ls build  
 classes ivy-reports test-classes testproject-1.1-distribution.zip test-reports  

The example provided in this article is specific to my testproject and Nexus installation/configuration. It can be downloaded here though.

Conclusion

Both the Nexus API and Apache Ivy provide means to retrieve artifacts from Nexus. Depending on your preference of scripting language, you can take either path or choose an alternative such as Maven for retrieving artifacts. When using Apache Ivy, you should be confident in your Ant scripting skills. When using the REST API, you can use shell scripting (probably using curl), Python, Perl or whatever else you like. My conclusion is that Nexus provides many options for retrieving artifacts. This helps in allowing a place for Nexus in almost every build process.