Java-based Scientific Data Objects

ABSTRACT:

The problem of portable data storage has been approached, attacked, and wrestled to the ground numerous times in the history of computing, but never totally subdued. With the advent of enormous computing efforts that are external to the scientific community, we believe it is time for a paradigm shift in the problem's approach.

Conventionally, specific data formats are designed and associated access mechanisms implemented. Extensions of these formats and porting efforts to other hardware and software platforms are an ongoing obligation of the supporting organization.

We believe that the actual data formats should be of concern only to the originator of the data and should not be specified in any manner; only an interface which allows access to the data. Portability, compression should be free.

Two months ago SUN released the Java Development Kit version 1.1, which had several new features used in our effort. These features can be used to provide a JAR file that contains not only the scientific data, but also the methods which provide access.

Only the Interface is standardised, the data itself can be stored in any format whatsoever, or may even be generated on the fly.

We have had access to the software for only a short time, but we have a very simple prototype running.

KEYWORDS:

Data Formats, Java

Co-Author: Joel Welling

This talk is online at http://www.psc.edu/~andrews

Introduction: the Pittsburgh Supercomputing Center

What is Java ?

System-independent object-oriented programming language. Lots of support/interest/religious fervour. Java compiled code is meant to be shipped between machines and run on a local JVM. Possibility of supplying both data and access methods.

Some new Java capabilities with JDK 1.1

• JAR file to hold both class definitions and methods
• Serialized Objects can be stored in files
• Digital signatures for security

How would Java data objects be different ?

Specify only an interface, i.e. form of methods that can be used to deliver, convert or display data. Interfaces can be implemented (form of multiple inheritance) and extended.

To java, code can be data, so class methods can be specified:

• In application (present method)
• By URL on web
• Locally as separate file
• Within Jar file with data (my favorite)

Advantages of this approach:

• Someone else does all the porting
• Excellent compression
• Digital signatures ensure data integrity
• Guarranteed extensibility without breaking existing code
• No restrictions whatsoever on how data is encoded
• Lots of work already proceeding on HPJava (High Performance Java)
• Can run within any JDK1.1 Net Browser

How to make a Jar file on a Supercomputer

• Port Java Virtual Machine to SC (very hard but good for soul)
• Generate data on SC, move to WS and run java (easy but kludgey)
• Modify GZIP utility on SC, copy java class from WS (so-so hard, so-so elegant)
• bring up DFS on SC and WS (easy and elegant).

Our Simple POP Demo

• Uses IEEE Cray data
• Simple 3D fluids data set
• Data Object was serialized and stored in file
• Data Object loaded from file and slices shown
• Access mechanisms are in data files, not application

Our prototype application

• You probably don't have a JDK 1.1 browser (yet)
• Even if you do, you probably accessed this file via the http protocol. To load classes from within the JAR file, we extended the ClassLoader class. It works fine from within an application, or if the file is loaded using the file protocol, but with http, the HotJava browser refuses to use an extended ClassLoader in case you're trying to bypass its security features

The ClassLoader works fine from an application (as I showed at the meeting) or if the file protocol is used for loading.

Click on vortices.jar to download the test jar file to your local machine.

If you're not using a JDK 1.1 browser, nothing will appear below.

We'll check into the security conflicts of using an extended ClassLoader with a browser and look for a workaround.