Announcement Notice (AN) 241.1 API (Web Services)

The image(s) below provides a preview of the of the most current AN 241.1 API (Web Service) Manual. The manual provides detailed instructions for using the 241.1 API and is updated periodically.

Page 1

Page 2

Figure 277402: Announcement Notice 241.1 Web Service for STI - Page 2

1.0 About

1.1 Announcement Notice 241.1

Announcement Notice (AN) 241.1 provides the Office of Scientific and Technical Information (OSTI) with the information needed to identify, process, announce and disseminate the results of work funded by the Department of Energy (DOE) or performed in DOE facilities (see DOE Order 241.1B). When submitted, the AN 241.1 set of metadata creates a citation/record that is processed by E-Link and then transferred into the appropriate databases for access (see search tools Search Tools).

see DOE Order 241.1B Search Tools

DOE sites can manually submit AN 241.1 metadata a single record at a time through a web interface on E-Link (www.osti.gov/elink). The AN 241.1 Web Service provides an additional option for submitting DOE's scientific and technical (STI) products to OSTI for dissemination. The STI product will be a full text document, such as a technical report, thesis or dissertation, book, journal literature, conference literature, or a program document. It could also be a multimedia file.

E-Link (www.osti.gov/elink)

AN 241.1 metadata and products submitted to OSTI, whether through a manual process or an automated process, are subject to the same requirements. A designated Releasing Official must ensure that each record has been reviewed for any content that would make it appropriate for release to the public. The Releasing Official is also responsible to ensure that full text documents posted on site or transmitted to OSTI do not contain Personally Identifiable Information (PII) (see https://www.osti.gov/elink/faq.jsp#PII). Use of the AN 241.1 Web Service does not remove these responsibilities from the Releasing Official.

https://www.osti.gov/elink/faq.jsp#PII

STI from DOE's research and development activities is disseminated through OSTI's databases such as OSTI.GOV, through multiagency databases such as Science.gov, through global databases such as WorldWideScience.org, and through arrangements with major Internet search engines such as Google.

OSTI.GOV Science.gov WorldWideScience.org

1.2 OSTI's AN 241.1 Web Service

OSTI's AN 241.1 Web Service allows a submitting site to send AN 241.1 information in a completely automated manner. Metadata is generated at the submitting site per the programming set-up by the site's IT developers and POSTed as an output file to OSTI. The schedule for the POST operation may be multiple times daily, once daily, once weekly, etc.

Records with unlimited access status and those with limited access can both be transmitted through the secure AN 241.1 Web Service. DOE organizations may establish a URL link to the unclassified, unlimited STI products on their websites and send the URL in the metadata or they may transmit the full text product to OSTI. All limited access (Controlled Unclassified Information or CUI) full text

Page 3

Figure 277403: Announcement Notice 241.1 Web Service for STI - Page 3

are required to be transmitted to OSTI and cannot be posted on the site's public web.

The AN 241.1 Web Service provides the ability to submit new Announcement records, edit/update records, or retrieve a record for review. A user manipulates a record by performing HTTP operations on the web service URL and providing XML metadata. The POST command/request allows a submitter to create or edit/update AN 241.1 records. The GET command/request allows a submitter to retrieve a record for review.

Submitters must coordinate with OSTI and submit records to a test environment before POSTing their first XML file for dissemination. The URL for Test E-Link is https://www.osti.gov/elinktest/2411api. OSTI's production web service for announcing STI is available at https://www.osti.gov/elink/2411api.

www.osti.gov/elinktest/2411api.www.osti.gov/elink/2411api.

2.0 Using the AN 241.1 Web Service

2.1 Credentials and Authentication

AN 241.1 transactions require an active E-Link user account. To request personal access to E-Link and obtain an active account, visit https://www.osti.gov/elink/register.jsp. To obtain an account for your client server to connect to OSTI's web service, contact E-Link Product Manager, Kim Buckner (bucknerk@osti.gov). Your web service account will be given a login name and password for Test E-Link. After testing is completed, you will be provided a login and password for production E-Link, as well as a separate password for your site's SFTP.

The AN 241.1 Web Service supports HTTP Basic authentication over SSL. With this method, the client connects to the HTTPS URL for either Test E-Link or, when testing is complete, for production E-Link. The POST and GET verb commands will pass along the standard Authentication HTTP header (base64 encoding).

www.osti.gov/elink/register.jsp.

2.2 POST (Create or Update) and GET (Retrieve) [Client to OSTI]

After successful authentication, the input body of the POST or GET request is read as an XML document, parsed, and loaded appropriately into E-Link. HTML status codes (200=OK, 401=Unauthorized, 500=System error, etc.) and an XML response, which includes certain metadata fields, are returned by OSTI's server to the client server to notify of either success in loading the record or failure.

The POST command/request is most often used to submit metadata for a new record that is being provided to OSTI for the first time, but it can also be used

Page 4

Figure 277404: Announcement Notice 241.1 Web Service for STI - Page 4

to edit or update an existing record that has previously been submitted. Submitters must "tell" the system which of the two activities they intend for a record. The OSTI ID is key to this. How and why is explained below:

When POSTING to an existing record in order to update it, the OSTI ID must be included in the XML record. The updated information will then overwrite the previously submitted information. If the OSTI ID is not included when trying to POST to an existing record, the Web Service believes you are wanting to create rather than update. It will create a duplicate record, but with a new, different OSTI ID. This situation always requires correction by logging into E-Link manually and manually deleting the duplicate record.

Metadata can be retrieved for any record previously submitted by your organization by using a GET request and supplying the OSTI ID as a parameter in the request. Authentication is required and is handled in the same fashion as a Create/Update command. The requested OSTI ID returns that specific XML record. A GET operation will not update a record, however.

When POSTING to an existing record in order to update it, the OSTI ID must be included in the XML record. The updated information will then overwrite the previously submitted information. If the OSTI ID is not included when trying to POST to an existing record, the Web Service believes you are wanting to create rather than update. It will create a duplicate record, but with a new, different OSTI ID. This situation always requires correction by logging into E-Link manually and manually deleting the duplicate record

2.3 Response and Notifications [OSTI to Client]

OSTI's server makes an immediate response when the client-side server sends a POST or a GET request. The primary goal of the response is two-fold: to let the client server "know" if it connected successfully and to provide a status message about the record that was POSTed or retrieved. The status says SUCCESS if the record correctly transmitted and the metadata loaded into E-Link. The status says FAILURE if the record was unable to load for any reason. The FAILURE response includes a message indicating the error or problem that kept the record from loading. Errors leading to a FAILURE response occur when a required metadata field is missing, when a site URL cannot be validated by the automated validation process, when a value is too long for the field it must load into, etc.

Note that the successful record returns its newly assigned OSTI ID. The response for the failed record will not have an OSTI ID since IDs cannot be assigned to records that never actually make it into E-Link

E-Link also generates an email containing the SUCCESS or FAILURE information about each submitted record. The site will be asked to provide to OSTI the names of the people to whom these automated emails will be sent.

It is the submitting site's responsibility to review the results of the web service transactions and to correct, then resubmit those records that initially failed.

2.4 Transmitting Full-Text Products to OSTI

A secure FTP directory is set-up at OSTI for each site utilizing the AN 241.1

Page 5

Figure 277405: Announcement Notice 241.1 Web Service for STI - Page 5

Web Service. A site cannot access any other site's SFTP directory...only the one assigned specifically to it. The process for providing both a metadata record and its corresponding full text product is as follows:

Site submits an XML metadata record to OSTI's E-Link system via the AN 241.1 Web Service.

The AN 241.1 Web Service notifies the submitting site that the XML record loaded successfully into E-Link and includes the newly assigned OSTI ID in the notification message. Notification goes out via server-to-server messaging and via an email to a site's designated distribution list.

A "script" written by the site's (client) software developers adds the new OSTI ID to the site's (client) database (records publishing system, review and approval system, etc.).

The "script" also reaches out to rename the corresponding full-text product, changing its file name from whatever it was originally to the new OSTI ID number. Only the number is used as the name, not any part of the OSTI ID text that sometimes prefixes the number. Example: a file named DOELAB-27834.pdf would be renamed 114073.pdf (if OSTI ID 114073 had been assigned to the corresponding metadata XML record).

Another automated command in the submitting site's (client) programming then transmits the file named 114073.pdf to the site's assigned SFTP directory.

The metadata record was submitted first and has been waiting in E- Link. It will not complete processing until E-Link recognizes that a document with the same OSTI ID is now present in the site's assigned SFTP directory.

If, on the first night after the XML record's submittal, E-Link does not detect a matching full-text product, the record stays in its "Submitted" state in E-Link. It will stay that way, as many days as it takes, until the full text product arrives from the site.

When E-Link recognizes that a full text product has arrived in the site's (client) directory and that the product has an OSTI ID matching a metadata record from the same site code, a siphon process takes place. The process attaches the product to the record.

After the "marriage," the record and its full text go through a "transfer" process that makes them available in the appropriate database. CUI records and documents are transferred to Science Research Connection (SRC) (see www.osti.gov/src), a password-protected system available

www.osti.gov/src),

Page 6