These are notes to accompany this part of the Summer School program. This section is intended to be on Sept. 14 1:30 - 3:15 pm. This is a play along section - you are encouraged to look at the code and do the examples as we go along. Please ask questions at any time.
Topics Covered :
- Utilizing VO Standards and Protocols: Client Side
Simple Cone Client
VO Standard
The first standard service adopted by the VO community was Simple Cone Search. In this tutorial we will show you have to build a java client to a cone service. First we will do the simple http get protocol, then a basic web service client.Java Class Path Setup
In order to perform these steps, you will need to have the java class paths setup. So perform the following:C:\nvoss\bin>setupjavaYou are now able to run the build scripts and client programs which are located in the nvoss\dev\coneclient directory, but first let's look at the basic client code. This assumes there is already some background and familiarity with the parameters of the VO cone search protocol, otherwise you can refer back to earlier slides and examples.
Http client code
In the following section of java code we will accept the required input parameters to a specific Cone Service example, GSC2. In this example we happen to be calling a jsp page but this will work for other basic http protocol services as well.
package coneclient;
import edu.jhu.pha.ivoa.*;
import java.io.*;
import java.text.*;
import java.net.*;
class ConeCaller {
public static void main(String[] args) throws Exception{
int ind =0;
String coneUrl="http://chart.stsci.edu/GSCVO/GSC22VO.jsp?";
double ra,dec,sr;
if (args.length >= 3 ) {
ra = Double.parseDouble(args[ind++]);
dec = Double.parseDouble(args[ind++]);
sr = Double.parseDouble(args[ind++]);
if (ind < args.length)
{
coneUrl = args[ind++];
}
callConeService(ra,dec,sr,coneUrl);
} else {
System.out.println("You need to supply ra,dec,sr,[coneURL] input paramaters.");
System.exit(1);
}
}
With the callConeService(), the java URL is constructed and the http get is invoked through the openStream().
// Construct URL and perform http get request
public static void callConeService(double ra, double dec, double sr, String coneUrl) throws Exception
{
URL cone = new URL(coneUrl + "RA="+ra+"&DEC="+dec+"&SR="+sr);
System.out.println("Calling cone URL: " + cone);
readVot(cone.openStream());
}
We then use a function like this readVot() to parse the VOTable stream and write out the first row. Several VOTable parsers are available, the more common ones with JAVA libraries are the SAVOT and JAVOT packages. These can be found in the ivoa/client directory. We do not go into a lot of detail with these examples, but several methods exist in these libraries for reading the VOTable data structures.
// Shows sample of how to parse stream from Cone service http get and
// output a row of the returned VOTable
public static void readVot(InputStream is) throws Exception{
// Using JAVOT VOTable parser
VOTWrap.VOTable vot = VOTWrap.createVOTable(is);
int resCount = vot.getResourceCount();
System.out.println (" has "+ resCount +" resources");
// could be more than one resource with more than one table - but this is a simple example
VOTWrap.Resource res = vot.getResource(0);
// making assumption there is a table in here
VOTWrap.Table tab = res.getTable(0);
int fcount = tab.getFieldCount();
System.out.println (" There are "+ fcount +" fields:");
for (int f=0; f < fcount; f++) {
VOTWrap.Field field = tab.getField(f);
System.out.print (field.getID()+":"+field.getUCD()+" ");
}
System.out.println();
// get one line out
System.out.println();
System.out.println("Println first row ..");
VOTWrap.TR row = tab.getTableData().getTR(0);
for (int f=0; f < fcount; f++) {
VOTWrap.TD td = row.getTD(f);
System.out.print (td.getPCDATA()+" ");
}
Run Http Client
You can run this client and the following tutorial programs by invoking the ant build.xml script file. If the build completed successfully you should see the BUILD SUCCESSFUL at the end. C:\nvoss\dev\coneclient>ant Now we want to run the coneclient to access the cone service and display the VOTABLE to show the return. Remember we need to input the ra, dec and sr parameters (in degrees). So we can use for example ra = 12, dec = 30, and sr = .1 C:\nvoss\dev\coneclient>java coneclient.ConeCaller 12 30 .1 The ant build script will also allow you to run the service by using the testhttp target (note the build script uses different input parameters) C:\nvoss\dev\coneclient>ant testhttp As an exercise you can replace the cone Service selected with another one you choose and rerun the client, or you can expand the code to accept an input URL.Web Service Cone Client
In this section we can expand the capability of the simple cone search by developing a web service client to use a cone web service. There are not many existing cone search web services available, so we will use one that we are familiar with, the SDSS. Note the difference in this example is we will use the build.xml file to perform a few critical steps. In order to build the web service client, you need to have access to the service WSDL (web service description language) file. This is normally accessible by the ?wsdl following the service call as shown in the build.xml file section below. In addition, you need to be able to generate the appropriate object classes in the client language (java in our case) and we can use Axis WSDL2Java tool to do this.- Note Axis was packaged with the course java files but normally would be installed independently along with Apache.
Client java class generation
<target name="axis" depends="">
<java classname="org.apache.axis.wsdl.WSDL2Java" fork="yes">
<arg value="http://skyserver.sdss.org/vo/dr2Cone/sdssConeSearch.asmx?wsdl"/>
</java>
</target>
Now in the main client code, coneWSClient.java, the generated classes are what is used to make the soap client call to the web service as follows.
ConeSearchWSLocator loc = new ConeSearchWSLocator(); // here you may switch url to a differnt endpoint ConeSearchWSSoap cs = loc.getConeSearchWSSoap();We can then do something very similar to what we did before with the simple cone client, but in the case of web services you need a service function which provides the correct object output, in this case, the SDSS web service returns a VOTABLE by the coneSearch method.
VOTABLE vot = cs.coneSearch(ra,dec,sr);Caution ** since the VO standards have been in the state of flux with schema (xsd) development, VOTABLE classes generated from wsdls have minor variations which may cause some service failures. In this example if you look carefully at the build script, you might notice a workaround to have VOTABLE class compatibility between packages. While web services ideally remove the language dependency on programmatic interfaces you still need to be aware of variability in implementations for xsd versions and also tools used to perform the wsdl generation and class generation. As shown above in the http example you can build the cone client running the ant script and then run the cone client web service. C:\nvoss\dev\coneclient>ant C:\nvoss\dev\coneclient>ant testWS The input parameters for ra, dec, and sr were included in the build.xml file for simplification in this tutorial.
<target name="testWS" depends="compile">
<java classname="coneclient.ConeWSClient" fork="yes">
<arg value="180"/>
<arg value="-1"/>
<arg value=".05"/>
</java>
</target>
The output is a portion of the VOTable returned from the SOAP call.
OpenSkyQuery portal Client
In this section we can show again another example of how to use the VO standard tools in a web service client. One of the significant advantages to the web services which return VO formatted output, is that you can programmatically build interfaces which give more flexibility in how you run queries, for example sweeping an area of the sky to perform cross matches. For this section we develop such as programmatic interface for the OpenSkyQuery and performs a cross match query as we have shown in the brown dwarf search student exercise using the OpenSkyQuery portal. In the directory nvoss\dev\skyportalclient you will see a similar pattern as what was shown above except the build.xml is tailored more to create a directory for all generated classes and also run a test. Again, Axis is used for converting the skyportal.wsdl to java classes which you find in the generated directory structure. The source file for this example is skyclient.java. In the main of this file we can build an OpenSkyQuery query as in the brown dwarf exerciseClient Query
// Sample Query for Brown Dwarf search
String qry = " SELECT o.objId, o.ra,o.dec, o.type,t.objId,t.j_m,o.z " +
" FROM SDSS:PhotoPrimary o, " +
" TWOMASS:PhotoPrimary t WHERE XMATCH(o,t)<2.5 " +
" AND Region('Circle J2000 16.031 -0.891 .10') " +
" AND( o.z- t.j_m)>1 " ;
if (ind < args.length) qry = args[0];
Client Web Service call
A slightly different way to invoke the web service, we actually include the endpoint address in the client code. The SkyPortalSoap has a method for sending a distributed query and returning a VOData object. This is not the same as a VOTABLE because it is a soap return, but we can cast this and go on as shown next.
// SOAP client code
SkyPortalLocator loc = new SkyPortalLocator();
loc.setSkyPortalSoapEndpointAddress("http://openskyquery.net/Sky/SkyPortal/SkyPortal.asmx");
System.out.println (" Using "+loc.getSkyPortalSoapAddress());
SkyPortalSoap sp = loc.getSkyPortalSoap();
VOData vod = sp.submitDistributedQuery(qry,"VOTABLE");
System.out.println (" Got data "+vod);
VOTable Serialization
Here is where we do the cast and ALSO serialize the VOTABLE using the Axis classes so we can write out an xml file. This would normally be packaged separately but since it isn't always obvious how to serialize the objects, it was included directly. Thanks to Ramon Williamson (NCSA) for helping to get the MessageContext for Axis working correctly.
VOTABLE vot = ((VOTableData)vod).getVOTABLE();
// CODE to SERIALIZE VOTABLE 1.1 using AXIS
FileWriter fw = new FileWriter("vot.xml");
Class javaType = vot.getClass();
MessageContext msgContext = new MessageContext(new AxisServer());
SerializationContext context =
new SerializationContextImpl(fw, msgContext);
context.setPretty(true); // formatting
context.setDoMultiRefs(false); // formatting
TypeDesc typedesc = vot.getTypeDesc();
QName qName = typedesc.getXmlType();
Serializer serializer = vot.getSerializer(Constants.AXIS_SAX, javaType, qName);
try
{
serializer.serialize(qName, null, vot, context );
fw.flush();
fw.close();
}
Use Client File Output in Visual Application
We now have the VOTable file (vot.xml) which we can perform analysis on with VO enabled applications. In this example, the VOTable version is 1.1 which is not fully compatible with the Mirage application. We can see the data by running TOPCAT or VOPLOT and doing a simple scatter plot. Again, assuming all the java paths were properly setup, simply start up your TOPCAT. C:\nvoss\skyportalclient>topcat In the Starlink TOPCAT, select File, the Load Table and browse for the vot.xml file. It will load in the application. You can then select on the scatter plot icon and select from the fields in the VOTable the X Axis and Y Axis variables. In this example of looking for Brown Dwarfs, you may want to look at the SDSS z vs. 2MASS j_m magnitude distributions. In your client you could also do various computations between columns, using the VOTable parser tools and export an additional column z-j_m for a color plot. VOPlot can be run similarly to load and inspect the VOTable results.SkyClient1 - adding variables for query
As in the above web service client, we can build a query string to the OpenSkyQuery SOAP request, yet we can extend the first SkyClient to take variable input such as ra and dec. This way you can query multiple fields and sample the sky….a common exercise. The other options would include loading a file list of coordinates but that is basic JAVA which will be left to you. We want to output a single VOTable so note also the declaration of RESOURCE, VOTABLE, and ArrayList before the loop through numFields. In our SkyClient1 class code we now have the variable coordinate inputs
// Search Set of coordinates step 0.001 deg
int numFields = 5;
double raInit = 16.030;
double decInit = -0.890;
double raArr[] = new double[numFields];
double decArr[] = new double[numFields];
// Declare Resource and VOTABLE which will be resource of All Queries
// i.e. creating a single VOTABLE
RESOURCE resource = null;
VOTABLE vot = null;
// Create ArrayList of Rows
ArrayList rows = new ArrayList();
for (int ic=0; ic<numFields; ic++)
{
raArr[ic] = raInit + 0.001*ic;
decArr[ic] = decInit - 0.001*ic;
// Sample Query for Brown Dwarf search
String qry = " SELECT o.objId, o.ra,o.dec, o.type,t.objId,t.j_m,o.z " +
" FROM SDSS:PhotoPrimary o, " +
" TWOMASS:PhotoPrimary t WHERE XMATCH(o,t)<2.5 " +
" AND Region('Circle J2000 " + raArr[ic] + " " + decArr[ic] + " .10') " +
" AND( o.z- t.j_m)>1 " ;
System.out.println (" Performing query: "+qry);
SkyClient1 SOAP
Then the SOAP locator and request are performed the same as the earlier example
// SOAP client code
SkyPortalLocator loc = new SkyPortalLocator();
loc.setSkyPortalSoapEndpointAddress("http://openskyquery.net/Sky/SkyPortal/SkyPortal.asmx");
System.out.println (" Using "+loc.getSkyPortalSoapAddress());
SkyPortalSoap sp = loc.getSkyPortalSoap();
VOData vod = sp.submitDistributedQuery(qry,"VOTABLE");
System.out.println (" Got data "+vod);
SkyClient1-fetch rows
Each time you loop through the coordinate set you want to add the VOTable data row to the overall VOTABLE
vot = ((VOTableData)vod).getVOTABLE();
resource = vot.getRESOURCE(0);
TABLE table = resource.getTABLE(0);
TR[] tr = table.getDATA().getTABLEDATA().getTR();
for (int i = 0; i < tr.length; i++)
{
rows.add(tr[i]);
}
System.out.println("Number of rows in array " + ic+ ": " + tr.length);
SkyClient1-combine rows
Build the cumulative VOTABLE by combining all the rows.
// Build VOTABLE of ALL Data by combining the rows
TR[] tr = new TR[1];
tr = (TR[])rows.toArray(tr);
System.out.println("Total Number of rows: " + tr.length);
resource.getTABLE(0).getDATA().getTABLEDATA().setTR(tr);
SkyClient1-serialize VOTABLE to file
The serialization is again performed as before and the VOTABLE file votBig.xml is generated. You can once again load and visually inspect the data in VOPlot, TOPCAT, etc.
FileWriter fw = new FileWriter("votBig.xml");
Class javaType = vot.getClass();
MessageContext msgContext = new MessageContext(new AxisServer());
SerializationContext context =
new SerializationContextImpl(fw, msgContext);
context.setPretty(true); // formatting
context.setDoMultiRefs(false); // formatting
TypeDesc typedesc = vot.getTypeDesc();
QName qName = typedesc.getXmlType();
Serializer serializer = vot.getSerializer(Constants.AXIS_SAX, javaType, qName);
try
{
serializer.serialize(qName, null, vot, context );
fw.flush();
fw.close();
}