Do you really want to completely clear your book?

API

General Access

To access the GlobalView API, you'll first need an Okta account. Once you've used that to log in, you'll receive an API key. You can use that API key to access the wiki using any mediawiki library.

For the purpose of this document, we'll use the python mwclient library referenced at: https://mwclient.readthedocs.io/

A simple test that your account is working follows, replacing the USERNAME and API_KEY sections with the ones provided: 

import mwclient
site = mwclient.Site('torque.leverforchange.org/', 'GlobalView/', scheme="https")
site.login("<USERNAME>", "<API_KEY>")
print(site.api('torquedataconnect', format='json', path='/competitions'))

The above should print out something like: 

OrderedDict([('result', ['LLIIA2020', 'LFC100Change2020', 'LFC100Change2017', 'EO2020', 'RacialEquity2030', 'Climate2030', 'ECW2020', 'LoneStar2020'])])

Using the python torquelcient

Download the client from pip: 

pip install torqueclient

The entire client is accessed from one toplevel option, Torque: 

from torqueclient import Torque

Logging in

You log in by instantiating the above object with your credentials. Follow the instructions in the General Access section above to get them. 

torque = Torque(
    "https://torque.leverforchange.org/GlobalView",
    "<USERNAME>",
    "<PASSWORD>",
)

Working with Competitions

Access all the competition information from the toplevel competitions field: 

torque.competitions

This returns an object that works both as something to index into, as well as something to iterate over. To see what competitions available to you, you can look at the keys() method: 

torque.competitions.keys()

Accessing a single competition works via subscripting: 

torque.competitions["LLIIA2020"]

But also, you can iterate over the competitions to do something with all of them: 

for competition in torque.competitions:
    print(competition.name)

Each competition will also provide a list of fields in the fields attribute, whic will be all the ones you have access to. The following will print out all the fields, in order, each on their own line: 

print("\n".join(sorted(torque.competitions["LLIIA2020"].fields)))

Working with Proposals

Like competitions above, you can access proposals through the proposals attribute. This will return an object that can be interated over or indexed into based on the proposal key.

Note: All data is loaded as lazily as possible. This means that the proposal data isn't loaded from the server until you try to access data within that proposal. Simply iterating over them will not lead to a server call.

The access looks like: 

lliia2020 = torque.competitions["LLIIA2020"]
proposals = lliia2020.proposals   # No server call

And to get all the keys available to you: 

lliia2020.proposals.keys()  # No server call

Looking at one: 

lliia2020.proposals["1956"]  # No server call

Looking at the data in one of them: 

lliia2020.proposals["1956"]["Executive Summary"]  # Server call!

Iterating over them: 

for proposal in lliia2020.proposals:
    print(proposal)["Application #"]  # Server call on each one!

As you can see above, you get proposal data out of them by accessing them via fields that are available to you (which you can check by competition.fields)

If you have permissions, you can also update the server data by setting the attribute on one. This will use the edit functionality and so is logged in the edit interface. 

proposal = lliia2020.proposals["1956"]
proposal["Executive Summary"] = "New Executive Summary"

Searching

Searching is currently a simple textual search, which can be done at the top level: 

proposals = torque.search("water in india")
print(len(proposals))

Or within a competition: 

proposals = lliia2020.search("water in india")
print(proposals[0]["Executive Summary"])

Search results are just a list of proposal objects, which are lazy loaded when access them. That way you don't need to load up all the data just to see how many results got returned. Then when accessing the data within them, calls will be made to the server.

Caching

torqueclient ships with two cache implementations, DiskCache (the default), and MemoryCache. These are available in the cache package. In addition, there's an empty Cache class which can be extended for custom caching.

The default DiskCache will store json objects in a subdirectory of .torqueclient in your home directory.

To use the MemoryCache (or any other cache), you need to instantiate the Torque object with it: 

from torqueclient.cache import MemoryCache
torque = Torque(
    "https://torque.leverforchange.org/GlobalView",
    "<USERNAME>",
    "<PASSWORD>",
    MemoryCache(),
)

See the inline documentation in the cache package for more information on the options in each cache, as well as how to implement your own.

Bulk Fetching

Because fetching proposal data from the server is i/o bound, something like the following can take a long time: 

proposals = lliia2020.proposals
for proposal in proposals:
    print(proposal["Executive Summary"])

It can be sped up by calling bulk_fetch on the top level Torque object. This will split out the calls to the server over multiple threads which speeds the process up significantly. This will pull from and populate the cache correctly, as well. Meaning doing bulk_fetch on cached objects will return instantly instead of going to the server.

You would use it in the above example by inserting it before the loop: 

proposals = lliia2020.proposals
torque.bulk_fetch(proposals)
for proposal in proposals:
    print(proposal["Executive Summary"])

bulk_fetch also accepts a list of proposals, as returned by search: 

proposals = torque.search("water in india")
torque.bulk_fetch(proposals)
for proposal in proposals:
     print(proposal["Executive Summary"])

Using mwclient

If you want to use the mwclient library to interface with mediawiki, the following documents how. This is also how you would interface if using a different language, replacing the python mwclient calls with the closest appropriate analogue in your language choice.

MediaWiki API

Because the torque API is fronted by mediawiki api, there are a few small changes from how APIs are generally created.

When calling the api, call the action 'torquedataconnect'. You can call with the format 'json' or 'xml', depending on what your library would like to use. For python, both get converted to an OrderedDict, and this document uses 'json'.

All responses are json objects with a single key "result", with the value being dependent on the endpoint, as described below. So for the following documentation, you can assume that you will always need to get response["result"]

Endpoint Reference

/competitions

Returns list of competitions available. Each competition is a String

Example python: 

response = site.api(
    'torquedataconnect',
    format='json',
    path='/competitions'
)

print(response["result"][0]) # Get the first competition

/competitions/<COMPETITION>

Returns the following json object: 

result -> {
  name: competition name, a string
  fields: fields available, a list of strings
  last_updated: ISO format of the last time any data in this competition was updated, a string
}

Example python: 

response = site.api(
    'torquedataconnect',
    format='json',
    path='/competitions/LoneStar2020'
)

print(response["result"]["fields"][0]) # Get the first field

/competitions/<COMPETITION>/proposals

Where COMPETITION is one of the available competitions returned by '/competitions' above.

Returns a list of all proposals ids available, which are strings. These are ordered in natural ordering for the competition (sometimes by organization name, sometimes by other fields).

Example python: 

response = site.api(
    'torquedataconnect',
    format='json',
    path='/competitions/LoneStar2020/proposals'
)

print(response["result"][0]) # The first proposal

/competitions/<COMPETITION>/proposals/<PROPOSAL_KEY>

Where COMPETITION is one of the available competitions, and PROPOSAL_KEY is one of the keys returned by .../proposals above

Returns a json object, where the keys are each fields (which are available as a list from the competition lookup above), and the values are the data for this proposal

Example python: 

response = site.api(
    'torquedataconnect',
    format='json',
    path='/competitions/LoneStar2020/proposals/410'
)

print(response["result"]) # Print the whole proposal!

/competitions/<COMPETITION>/proposals/<PROPOSAL_KEY>/fields/<FIELD>

Where COMPETITION, PROPOSAL_KEY are similar to above, and FIELD is one of the field names return as above.

Returns a string for that specific field in this proposal, in this competition.

Example python: 

response = site.api(
    'torquedataconnect',
    format='json',
    path='/competitions/LoneStar2020/proposals/410/fields/Application #'
)

print(response["result"]) # Prints 410

/search

Takes in a secondary parameter, "q" which is the query for searching. Returns a list of URIs that represent proposals for which this query matchines. The order is in descending order of relevance. 

response = site.api(
    'torquedataconnect',
    format='json',
    path='/search',
    q="water in india"
)

top_search_result_uri = response["result"][0]
top_search_result = site.api(
    'torquedataconnect',
    format='json',
    path=top_search_result_uri
)["result"]
print(top_search_result) # Outputs the whole result

/competitions/<COMPETITION>/search

The same as above, but for a specific competition

Example python: 

response = site.api(
    'torquedataconnect',
    format='json',
    path='/competitions/LoneStar2020/search',
    q="water in india"
)

top_search_result_uri = response["result"][0]
top_search_result = site.api(
    'torquedataconnect',
    format='json',
    path=top_search_result_uri
)["result"]
print(top_search_result) # Outputs the whole result
Retrieved from "https://torque.leverforchange.org/documents/index.php?title=API&oldid=251"