The Torture Database API
The Torture Database is the largest public repository of primary-source documents relating to the Bush administration’s interrogation policies. The database came out of a Freedom of Information Act (FOIA) request filed in 2003, as part of our ongoing effort to uncover how our government used torture in the years following September 11, 2001. Following a court ruling in our favor, the government released more than 130,000 pages of documents on the interrogation program, including memos detailing the techniques authorized by the CIA, autopsy reports of detainees who died in custody, authorizations for the use of torture from the highest levels of government, email threads documenting heroic efforts to stop the use of torture, and more.
The Torture Database API provides access to documents stored at thetorturedatabase.org. It is a RESTful API built on HTTP and:
- Uses built-in HTTP capabilities for passing parameters
- Has search features comparable to thetorturedatabase.org site
- Returns data in JSON or XML formatting
The API was rapidly developed using the Drupal Services module, which provides web service callbacks for data as represented by the Drupal content management system. As such, the structure of the data returned by this API bears a close resemblance to standard Drupal objects.
Formatting Options
The API currently provides two data formatting options, which is determined by the last component in the URL:
JSON formatting:
http://www.thetorturedatabase.org/rest/searchnode/retrieve.json
XML formatting:
http://www.thetorturedatabase.org/rest/searchnode/retrieve
Base URL
All API URLs referenced in this documentation start with one of the following base parts:
To retrieve a list of items for a specific content type (e.g., document):
http://www.thetorturedatabase.org/rest/getnode/retrieve
To retrieve document search results using keywords and filters:
http://www.thetorturedatabase.org/rest/searchnode/retrieve
To retrieve the fields for a specific node by a numeric ID (a node is Drupal term meaning a piece of content on the website):
http://www.thetorturedatabase.org/rest/fullnode/retrieve
Authentication
Since no authentication is required, please be cautious of the number of API calls being made within a short time period. Aggressive use of the API may adversely affect the thetorturedatabase.org site and may lead to the removal of the API for all users.
How to retrieve a list of all data for a certain type
List of data objects: Agency, Authors/Recipients, Document Types, Techniques, Detainee, Document, Incident, Location, Official, Source
Note: By default, only the first 100 results will display for these requests. The page parameter should be passed to view remaining results (e.g. http://www.thetorturedatabase.org/rest/getnode/retrieve?name=[type]&page=1). page=0 by default.
Agency (primary key: tid)
http://www.thetorturedatabase.org/rest/getnode/retrieve?name=agency
Authors/Recipients (primary key: tid)
http://www.thetorturedatabase.org/rest/getnode/retrieve?name=Authors/Recipients
Document Types (primary key: tid)
http://www.thetorturedatabase.org/rest/getnode/retrieve?name=Document%20Types
Techniques (primary key: tid)
http://www.thetorturedatabase.org/rest/getnode/retrieve?name=Techniques
Detainee (primary key: nid)
http://www.thetorturedatabase.org/rest/getnode/retrieve?name=detainee
Document (primary key: nid)
http://www.thetorturedatabase.org/rest/getnode/retrieve?name=document
Incident (primary key: nid)
http://www.thetorturedatabase.org/rest/getnode/retrieve?name=incident
Location (primary key: nid)
http://www.thetorturedatabase.org/rest/getnode/retrieve?name=location
Official (primary key: nid)
http://www.thetorturedatabase.org/rest/getnode/retrieve?name=official
Source (primary key: nid)
http://www.thetorturedatabase.org/rest/getnode/retrieve?name=source
How to retrieve data for a specific node
A node is a piece of content on the website with nid as the primary key. Each node and its fields can be retrieved in full.
Pass the ID of the node (e.g. 6822):
http://www.thetorturedatabase.org/rest/fullnode/retrieve?nid=6822
The fields returned are all of the custom fields associated with each content type, in addition to some standard Drupal metadata. We’ve attempted to have each field name be self-documenting but please contact thetorturedatabase@aclu.org if the purpose of a field is unclear. References to other objects are done with a numeric ID, and are usually clearly marked with a nid key. References to objects without a nid primary key are marked with a value key and refer to an object with a unique tid primary key. Objects with a tid primary key cannot be retrieved with the fullnode call, as they have no additional fields other than what is provided in the getnode call.
How to search by keyword(s)
The Torture Database API allows you to search the database using similar options as provided through the web interface.
To search by keyword, pass the 'keys' parameter with the searchnode base URL:
http://www.thetorturedatabase.org/rest/searchnode/retrieve?keys=john%20smith
Note: Since results are viewed 10 items at a time by default, a page parameter can be set to paginate through the results:
http://www.thetorturedatabase.org/rest/searchnode/retrieve?keys=john%20smith&page=2
How to filter keyword results by date, category, and advanced search options
Search filters use the same formatting that is passed to the Apache SOLR search engine. Since the structure of these filters can be opaque, the easiest approach is to copy the filter values from the actual site when using the search/advanced search options. You can copy the part of the URL after ‘filters’ to find the relevant filter options. For example:
Appended to the base URL for searching:
For reference, here are some specific search options:
Filter by Date
Document Date
tds_cck_field_doc_date:[2006-01-01T00:00:00Z TO 2012-01-01T00:00:00Z]
http://www.thetorturedatabase.org/rest/searchnode/retrieve?keys=john%20smith&filters=tds_cck_field_doc_date%3A[2006-01-01T00%3A00%3A00Z%20TO%2020012-01-01T00%3A00%3A00Z]
Specific Incident of Abuse
tdm_cck_field_incident_date:[2003-05-01T00:00:00Z TO 2006-05-02T00:00:00Z]
Release Date
tds_cck_field_doc_release_date:[2005-05-01T00:00:00Z TO 2010-05-02T00:00:00Z]
Filter By Category (for multiple values, separate by comma)
Authoring Agency
sm_cck_field_doc_from_agency:35
Officials mentioned
im_cck_field_doc_officials:1700
Detainees Mentioned
im_cck_field_doc_detainees:981
Methods Mentioned
sm_cck_field_techniques:100
Document Type
sm_cck_field_doc_type:649
Advanced Search
Author By Name
sm_cck_field_doc_from:1117
Authoring Agencies
authoring_agencies=31
Recipient By Name
sm_cck_field_doc_to:584
Receiving Agencies
receiving_agencies=31,39
Mentions the Following
im_cck_field_doc_detainees:646
Incident Location
im_cck_field_incident_location:148
Regions
regions:Afghanistan
regions:Iraq
regions:Guantanamo
regions:Everywhere Else