Submitting Condor Jobs to UNICORE
This technical note explains how to submit Condor jobs to a UNICORE VSite. It is similar in character to our Condor-GT4 description in that it elaborates Condor's official documentation (which happens to be almost non-existing for UNICORE). However, unlike Condor-GT4, we have not tested the Condor-UNICORE interface - we only got a basic test running - so we cannot recommend it or provide additional details.
Create a keystore
Unlike Globus Toolkit, UNICORE doesn't use proxy certificates for authentication. Instead, the client (= Condor) needs access to your Grid user certificate, which has to be stored in a format known as JKS keystore. The same keystore file has to also contain trusted CA certificates. If you only have the PEM-encoded user certificate and private key, like with Globus, creating the keystore is a two-step process
Step 1: Convert usercert.pem and userkey.pem to usercert.p12
openssl pkcs12 -export -in usercert.pem -inkey userkey.pem -out usercert.p12
Step 2: Create the keystore usercert.jks
Download UNICORE's GPE application client (GUI, requires Java).
When you first start the client, it will create an empty keystore file and ask you for a new password to protect it.
After that, open Settings/Keystore Editor. Select File/New, which will allow you to create a new keystore file in a location that you specify (e.g. usercert.jks). Then open Actions/Import Keystore... and point to the usercert.p12 created in the previous step.
Next, open Actions/Import Certificate... and in the following dialog enter the directory where your CA certificates are stored. Select all CA certificates to be imported (in D-Grid, the certificates are stored in cryptically named files with .0 extension, e.g. dd4b34ea.0).
Finally, use File/Save to create the keystore.
Step 3: Verify the usercert.jks contents
You can check the contents of usercert.jks using the program
jploski@srvgrid01:~/.globus> keytool -list -keystore usercert.jks Enter keystore password: Keystore type: JKS Keystore provider: SUN Your keystore contains 2 entries jan ploski's dfn-verein id, Oct 9, 2008, PrivateKeyEntry, Certificate fingerprint (MD5): BA:C5:99:AF:2F:3D:49:4B:DC:AF:48:5D:EA:17:36:D8 dfn-verein pca grid - g0, Oct 9, 2008, trustedCertEntry, Certificate fingerprint (MD5): 41:39:4A:58:2E:F0:45:B2:29:28:F1:72:AB:F7:05:08
Create a job command file
Here is an example job command file for submitting a simple job to a UNICORE VSite:
executable = /bin/bash arguments = hello.sh transfer_input_files = hello.sh,hello_in.txt transfer_output_files = hello_out.txt transfer_executable = false whentotransferoutput = ON_EXIT notification = NEVER universe = grid grid_resource = unicore zam177.zam.kfa-juelich.de:4009 FZK-DGIREF-NJS output = hello.out error = hello.err log = hello.log keystore_file = /home/jploski/.globus/usercert.jks keystore_alias = jan ploski's dfn-verein id keystore_passphrase_file = /home/jploski/.globus/passphrase queue
The UNICORE-related parameters are:
For D-Grid, the gateway address(es) can be looked up here: http://www.fz-juelich.de/jsc/unicore/unicoreSites_D-Grid.xml
Note that as of the time of writing D-Grid installations contain UNICORE 5. We have not tested Condor job submission to UNICORE 6.
The hello.sh script used above contains:
#!/bin/bash echo Hello, world cat hello_in.txt date uname -a cp hello_in.txt hello_out.txt
And the hello_in.txt file contains:
This the content of a text file transferred to the execution site (hello_in.txt).
Fix ugahp.jar in Condor
We had problems with the out-of-the-box UNICORE adapter shipped with Condor 7.0.1 in condor/lib/ugahp.jar. Specifically, the submitted job script was rejected by the PBS adapter at the VSite because of a missing email address. We fixed this problem by patching and recompiling the source code for ugahp.jar.
Submit your job
The detailed protocol of execution can be seen in GridmanagerLog.<your username>.
In our test, we got the following output files:
jploski@srvgrid01:~/condor-unicore> cat hello.err id: cannot find name for group ID 64003 This script was created and executed by Unicore UNICORE - start of user output on stderr UNICORE - end of user output on stderr UNICORE EXIT STATUS 0 + jploski@srvgrid01:~/condor-unicore> cat hello.out Hello, world This the content of a text file transferred to the execution site (hello_in.txt). Do Okt 9 12:53:58 CEST 2008 Linux wn10 2.6.9-55.0.12.ELsmp #1 SMP Thu Nov 1 09:20:43 CDT 2007 x86_64 x86_64 x86_64 GNU/Linux jploski@srvgrid01:~/condor-unicore> cat hello_out.txt This the content of a text file transferred to the execution site (hello_in.txt).
You can also download the GridmanagerLog file created by running this test.