File processing using a Spring component

Introduction

The Oracle SOA Suite 11g FileAdapter provides (among other things) functionality to read a variety of text file formats such as CSV and transform them into XML. This makes it easier to use them in for example BPEL. Usually the FileAdapter provides enough functionality to correctly parse files. Sometimes however, this is not possible. An example of such a file;

h1;h2;h3;h4
A;B;C;D
E;F;G;H
Total: 2

h1 to h4 are header fields. The second and third line are the records and the last line indicates how many records are present in the file.

The last line cannot be excluded from parsing by the FileAdapter and will lead to errors. As a workaround and to provide some flexibility in file parsing, the following solution can be implemented. Keep in mind that this solution will cause problems for large files since the file will in memory be converted to XML.

Read the file as opaque using the FileAdapter. Create a Java class which has a public method with a byte array as input parameter and some parsing parameters. Wrap the Java class as a Spring component. Then you can use the component in a BPEL process as JDeveloper will generate XSD's/WSDL's for the Spring component for you.

Implementation

FileAdapter

In BPEL define a file adapter and select not to use a schema;

Java class parsefile

A Java class does the parsing. You can of course expand this example. Currently empty lines are also processed as entries, which might not be what you want. This class is also specific for the example given in the introduction (but is flexible enough for related file formats). I have chosen to return an ArrayList of ArrayLists. The first (outer) ArrayList contains the records of the file and the inner ArrayList contains the items in the record separated on a separator regular expression. The header lines can be ignored and you can specify to skip lines starting with a specific string. If you want to be able to skip multiple strings (in order to make a selection in the file of lines to process), you can change the type of the last parameter of the parse function to ArrayList and alter the code to loop over that.

Create the following Java class;

package nl.smeets.myfilereader;

import java.io.BufferedReader;
import java.io.ByteArrayInputStream;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.io.InputStreamReader;

import java.util.ArrayList;

public class parsefile {
    public parsefile() {
        super();
    }

    public ArrayList<ArrayList<String>> parse(byte[] bytes_in,
                                              String separator,
                                              Integer ignorefirstlines,
                                              String ignoreStartsWith) {
        ArrayList<ArrayList<String>> l_retval =
            new ArrayList<ArrayList<String>>();
        ByteArrayInputStream ba_in = new ByteArrayInputStream(bytes_in);
        BufferedReader br_in =
            new BufferedReader(new InputStreamReader(ba_in));
        Integer linecounter = 0;
        String line = null;
        try {
            while ((line = br_in.readLine()) != null) {
                if (!line.startsWith(ignoreStartsWith) &&
                    linecounter >= ignorefirstlines) {
                    l_retval.add(parseLine(line, separator));
                }
                linecounter++;
            }
        } catch (FileNotFoundException ex) {
            ex.printStackTrace();
        } catch (IOException ex) {
            ex.printStackTrace();
        } finally {
            //Close the BufferedReader
            try {
                if (br_in != null)
                    br_in.close();
            } catch (IOException ex) {
                ex.printStackTrace();
            }
        }
        return l_retval;
    }

    ArrayList<String> parseLine(String input, String separator) {
        ArrayList<String> retval = new ArrayList<String>();
        String[] splitVals = input.split(separator);
        for (String addVal : splitVals) {
            retval.add(addVal);
        }
        return retval;
    }

    public static void main(String[] args) {
        //test; skips first line and all lines starting with Total. uses ; as separator
        //parsefile parsefile = new parsefile();
        //System.out.println(parsefile.parse("h1;h2;h3;h4\ne1;f1;g1;h1\ne2;f2;g2;h2\nTotal: 1".getBytes(), ";", 1, "Total"));
    }
}

Extract interface

Right-click the Java file, Refactor, Extract interface

Spring bean

Use the wizard to create a Spring bean and define it as follows;

<?xml version="1.0" encoding="windows-1252" ?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:util="http://www.springframework.org/schema/util"
       xmlns:jee="http://www.springframework.org/schema/jee"
       xmlns:lang="http://www.springframework.org/schema/lang"
       xmlns:aop="http://www.springframework.org/schema/aop"
       xmlns:tx="http://www.springframework.org/schema/tx"
       xmlns:sca="http://xmlns.oracle.com/weblogic/weblogic-sca"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.5.xsd http://www.springframework.org/schema/util http://www.springframework.org/schema/util/spring-util-2.5.xsd http://www.springframework.org/schema/aop http://www.springframework.org/schema/aop/spring-aop-2.5.xsd http://www.springframework.org/schema/jee http://www.springframework.org/schema/jee/spring-jee-2.5.xsd http://www.springframework.org/schema/lang http://www.springframework.org/schema/lang/spring-lang-2.5.xsd http://www.springframework.org/schema/tx http://www.springframework.org/schema/tx/spring-tx-2.5.xsd http://www.springframework.org/schema/tool http://www.springframework.org/schema/tool/spring-tool-2.5.xsd http://xmlns.oracle.com/weblogic/weblogic-sca META-INF/weblogic-sca.xsd">
 
<bean name="ParseFileBean" class="nl.smeets.myfilereader.parsefile"/>
<sca:service name="ParseFileBeanService" target="ParseFileBean" type="nl.smeets.myfilereader.Iparsefile"/>
</beans>

First make sure the Java interface and class are compiled to avoid errors such as the one below;

Now you can use the composite editor to expose the Spring bean or use it in a BPEL process.

BPEL

Use BPEL to bring it all together. In BPEL you can assign the parameters of the parsing Java method. In this example I've hardcoded them in the process but of course you can also use BPEL preferences (described in a previous post; http://javaoraclesoa.blogspot.com/2012/02/changing-properties-of-bpel-process-at.html) which can be defined in the composite.xml file and modified at runtime so if the format of the file changes, you don't have to redeploy the BPEL process. Don't forget to update the configuration plan supplied to reflect your local environment. *.txt files from (logical path) READ_FILE_DIR are read and xml files are written as output to logical path WRITE_FILE_DIR.


Input;

h1;h2;h3;h4
A;B;C;D
E;F;G;H
Total: 2

Output;

<?xml version="1.0" encoding="UTF-8" ?>
<parseResponse xmlns="http://myfilereader.smeets.nl/">
   <return xmlns:ns0="http://myfilereader.smeets.nl/types" xmlns="">
      <item>A</item>
      <item>B</item>
      <item>C</item>
      <item>D</item>
   </return>
   <return xmlns:ns0="http://myfilereader.smeets.nl/types" xmlns="">
      <item>E</item>
      <item>F</item>
      <item>G</item>
      <item>H</item>
   </return>
</parseResponse>



You can download the entire example here; https://www.dropbox.com/s/ytgwjz7zjlvscmj/DemoReadFile.zip?dl=0