In this article I'll demonstrate how these left-over composite parts can be identified and removed on an 11.1.1.7 SOA environment. First try this procedure on a development or test environment before executing it in production! This method is not supported by Oracle (or me) in any way and using it is entirely at your own risk. If something breaks, tell me so I can update this article. Thanks!
Please mind that these actions, although they help with the start time and memory usage of your SOA environment, have less impact on run-time performance than for example purging of instances and reducing the amount of deployed composites (or tweaking datasources, soa-infra database, JVM, etc).
 |
| SOA Suite can be up quickly! |
Introduction
What is a composite in the MDS?
WebLogic Server first brings up its Managed SOA server. After this managed server is up and RUNNING, the soa-infra application is started. After the soa-infra application is started, the composites are loaded from the MDS and put in memory.
There is a folder called 'deployed-composites' in the root of the soa-infra MDS. If you are an experienced Oracle SOA developer or administrator, you have probably encountered this folder before. For example when the soa-infra application does not want to start, you can remove an entry in deployed-composites.xml which is in the deployed-composites folder (see here. you can even do this offline (here)). A composite however is not only an entry in the deployed-composites.xml file. There is also a separate folder which contains the files of which the composite consists. These are mostly XML and JAR files.
Cleaning
Entries in the deployed-composites.xml file are visible in the Enterprise Manager. Entries which are not in the deployed-composites.xml file but do have a folder in the MDS, are not visible, but are loaded at server start. This increases startup time and memory usage.
Deployed-composites
First create an MDS export. This is described here. When you have this export, you can extract it and execute the following Python 2.7 script (don't forget to update the paths for your environment and the SOA server name). What the script does is look at the deployed-composites.xml file and identify all entries in the default partition of the deployed-composites folder which do not have an entry in deployed-composites.xml. Next it creates a WLST script to remove these entries. The WLST script which is created, should have a 'connect' statement at the top and should be executed against your SOA (managed) server. You can also execute it remotely. Make sure you pick the correct wlst.sh or wlst.cmd to start a WLST interpreter which has deleteMetadata available. deleteMetadata was the only WLST command I found which allows deleting documents outside the 'apps' MDS folder.
import os
import xml.etree.ElementTree as ET
import xml.dom.minidom as minidom
import sys,re
import argparse
deployed_composites="D:\\tmp\\soa-infra_metadata\\deployed-composites\\deployed-composites.xml"
#read a file and return a ElementTree
def get_tree_from_xmlfile(filename):
if os.path.isfile(filename):
tree = ET.parse(filename)
return tree
else:
raise Exception('Error opening '+filename)
def get_deployedcomposites(deployed_composites):
deployed_composites_root = deployed_composites.getroot()
return deployed_composites_root
deployed_composites_el = get_tree_from_xmlfile(deployed_composites)
deployed_composites_root = deployed_composites_el.getroot()
list = []
for rev in deployed_composites_root.findall('composite-series/composite-revision'):
list.append(rev.attrib.get('dn').replace('!','_rev'))
for dir in os.listdir('D:\\tmp\\soa-infra_metadata\\deployed-composites\\default'):
if 'default/'+dir not in list:
print "deleteMetadata(application='soa-infra',server='soa_server',docs='/deployed-composites/default/"+dir+"/**')"
Directories
I did not find a way to delete directories outside the apps folder by using WLST commands. Luckily the Oracle A-Team also realized this and wrote some custom Java code to address this issue. See here. The blog article offers a project which you can download which contains sample code on how to programatically do interesting things to the MDS. For this specific use-case I updated the code to look for directories which do not contain documents. These directories recursively removed (that's why I have to catch a NullPointerException. the directory could already be gone). Of course first check if this code works for you before actually deleting directories. I only updated the work method in the sample code provided by Oracle which you can download here.
public static void work()
{
try
{
MDSInstance mdsInstance = null;
if (true)
{
mdsInstance = MDSUtils.initializeDBStore("soa_mds_user",
"soa_mds_pass",
"jdbc:oracle:thin:@hostname:port/sid",
"soa-infra",
MDS_CONNECTION_NAME);
}
Boolean foundDocument = false;
// Find a resource
if (true)
{
List<ResourceName> list = MDSUtils.findResource(mdsInstance, "/deployed-composites/", false);
List<ResourceName> listinner = null;
List<String> path = new ArrayList<String>();
List<Boolean> type = new ArrayList<Boolean>();
System.out.println("List: (" + list.size() + " element(s))");
for (ResourceName rn : list) {
path.add(rn.getAbsoluteName());
type.add(rn.isPackageName());
}
ResourceName rn = null;
for (int i = 0; i < list.size() ; i++) {
rn = list.get(i);
foundDocument = false;
if (rn.isPackageName()) {
for (int j = 0 ; j<path.size();j++ ) {
if (path.get(j).startsWith(rn.getAbsoluteName())) {
if (!type.get(j)) {
foundDocument = true;
break;
}
}
}
if (!foundDocument) {
try {
MDSUtils.deleteResource(mdsInstance, rn);
} catch (Exception e) {
System.out.print("");
}
System.out.println("Deleted: "+rn.getAbsoluteName());
}
}
}
}
System.out.println("Done");
}
catch (Exception e)
{
e.printStackTrace();
}
}
Be careful with this code if you use other things inside the MDS than documents and packages (folders) such as customizations.
The MDS schema
Everything which was present in the MDS once, but has been deleted, remains in the MDS database. You can confirm this by looking at the SOA_MDS schema and browse for items you thought had been deleted in the MDS_PATHS table. Deleted items can be identified by checking MDS_PATHS for entries which do not have a PATH_HIGH_CN which is null. See here. Deleted documents however are not loaded by the server during startup, but if you once had a lot of stuff in the MDS, the below procedure might be worthwhile since MDS queries will perform faster with less data.
Manually 'fiddling' with the data in the SOA_MDS tables is not recommended and dangerous since there are no foreign key constraints but there are references from other tables! (with column names which do not easily allow you to identify the links between data in the different tables). As a workaround, you can create an MDS export (this only takes the 'not-deleted' documents and folders), backup the SOA_MDS schema, truncate tables containing documents/folders (not the partitions and similarly important stuff!) and re-import the MDS export. You can also recreate the SOA_MDS schema completely with the RCU (Repository Creation Utility) before the import. This should provide you with a clean MDS. As indicated before, do test this procedure to make sure you have identified the correct SOA_MDS tables! There are also several other ways to tweak MDS performance. See here.
No comments:
Post a Comment