Automatic Data Transfer Services

Follow

At Scope 5, we recognize that collecting and maintaining high integrity data is one of the greatest challenges of many sustainability programs. To assist you, our clients, in the work of data maintenance, we offer ADT or automatic data transfer services. This article is about the various forms of ADT that we offer.

 

Types of ADT Sources

ADT services provide data exchange between a client's Scope 5 account and a specific data source. Note that an ADT service can both pull data from a data source and push data to a data source. There are two types of ADT services:

  • Standard ADT Service.
  • Advanced ADT Service.

Setting up any of these types of ADT services starts with a conversation with your Scope 5 account manager. Your account manager will work with you to understand the type of service that will best meet your needs and will work with the Scope 5 technical team to configure and enable ADT for your account. The two types of ADT services are described in the following paragraphs. The focus will be on Standard ADT Services, as this is the most commonly used type of ADT service. 

 

Standard ADT Service

Standard ADT services include:

  • Pulling data from conventional utility accounts that are accessible online via a standard web browser.
  • Pulling data from paper records, images of paper records or electronic documents such as spreadsheets.
  • Pulling/pushing data from/to any data source that is accessible online via a standard web browser.

In this article we describe step-by-step the setup process for conventional utility accounts. The same process is adapted to the two other Standard ADT services. For more information, please contact your account manager.

Conventional utility accounts are likely familiar to most readers in the form of residential electricity, natural gas or water/sewer accounts. Scope 5 ADT routinely pulls usage (and optionally, cost) data on behalf of our clients from the commercial equivalent. In order to do so, the utility account must be accessible via the Internet. ADT services for conventional utility accounts can only pull data from the account - it is not possible to push data to a conventional utility account.

Setting up a Conventional Utility ADT

There are several steps in setting up ADT between a client's Scope 5 account and a specific utility provider. These are outlined below.

1. Identifying the utility provider - as part of your initial conversation with your account manager, you will identify the utility providers from which you wish to source data using ADT. Your account manager will then make those providers generally available to your Scope 5 account.

2. Connecting a tracker to the utility provider - in the next step, you will identify the specific tracker(s) to which the ADT service should store the data that it pulls from the utility provider. You will see the following option on each tracker's edit form:

adtform1.png   

Select the specific utility provider that you wish to connect to the tracker:

adtform2.png

3. Account website information - upon selecting the provider, you will be prompted to enter the utility account website and credentials for the account:

adtform3.png 

4. Additional utility account information - occasionally, your utility provider's website URL and your organization's account credentials are sufficient to uniquely identify the specific utility account from which the ADT agent should source data. Often, additional information (such as account number or meter number) will be required to disambiguate the intended data source. Depending on the utility provider and your utility account, the fields for providing this additional account information may be available immediately on selecting the utility provider and you can use them to map the subject tracker to its intended data source. Often, your Scope 5 account manager will work with you to determine how to map specific trackers to specific sub-accounts or meters. You may be asked to return to this form once more to provide additional account information.

The First Data Transfer

At this point you will have provided all the necessary information to access your data from the utility provider. The first data transfer may commence immediately but may take up to 10 days. 

Depending on your Scope 5 account's baseline date and the availability of historical data from your utility provider, the first data transfer may pull multiple historical records. Often, clients will see as much as two or three years of utility data pulled in the first data transfer.

Subsequent Data Transfers

Following the initial data transfer, Scope 5's ADT agent will periodically access your utility provider's account, looking for new data. Over time, the agent will 'learn' the frequency at which it can expect to find new data and will connect to the provider's website at roughly that frequency. (There is a limit on the maximum rate at which the agent will look for new records). Each time new data is found, the ADT agent will transfer that data and will 'reset the clock', remaining dormant until the next new data is expected. 

Note that in the case of ADT from conventional utilities, historical data is assumed to remain constant. The ADT agent will not automatically examine historical data to look for changes. This is in contrast to the other types of ADT services which may periodically reconcile entire account history. If a historical account reconciliation is required, you can ask your account manager to initiate such a reconciliation.

Assumptions

When implementing ADT from conventional utility accounts we expect that:

  1. Usage (and cost if needed) are easily identified, from a single location, electronically. 
  2. All bills for a single account conform to a consistent format.
  3. No data conversion (such as unit conversion, summing across multiple records, etc.) is required.
  4. Data will be pulled at a frequency no greater than once each month.

We are happy to handle exceptions to these rules on a case-by-case basis. Doing so may mean that a specific ADT data source is recategorized as a specialized data platform (see the next section) and may incur additional cost.  

 

Advanced ADT Service

All data sources not conforming to the Standard ADT Services parameters, fall into the Advances ADT Service category. Advanced ADT Services comprise two sub categories:

  • Existing Services Available.
  • Custom Advanced ADT.

Existing Services Available

Scope 5 ADT services are available for a number of specialized platforms. Examples include Portfolio Manager, Tableau and Concur Travel Services. If there is a specific, generally available data platform that you would like to connect your Scope 5 account to using ADT services, speak with your account manager. There may be an existing Scope 5 ADT service for that data platform or Scope 5 may consider creating one.

Specialized ADT Functionality

Most ADT services for specialized data platforms offer specialized functionality. Several examples include:

  • push functionality - in which data can be pushed from Scope 5 to an external data platform
  • sync functionality - in which the ADT agent combines push and pull functionality to keep a remote data platform and a Scope 5 account synchronized across time (this type of service includes maintenance of historical data) 
  • data routing functionality - in which data from a specialized platform can be routed to one of many Scope 5 trackers based on attributes of the data 

The figure below illustrates the kind of variation on the ADT edit form that you might expect to see for a specialized ADT service:

specialadt.png

This particular example is for the Portfolio Manager ADT service. Note the inclusion of a field to identify a particular Property ID as well as the options to sync/push and to handle bulk delivery. The bulk delivery option is used for data that is generally a-periodic (as opposed to conventional utility data that is generally periodic in nature).  

Custom Advanced ADT

From time to time, we are asked to write ADT agents for custom data repositories such as a customer's ERP system or other specialized databases. These usually require custom work from the Scope 5 software development team.  

 

ADT Service Implementation and Guiding Principles

We strive to provide all data transfer services electronically. At times we may complement these electronic services with manual services. For example, it may take us some time to write an electronic agent for a specific data source, especially if the source is specialized. In this case, we may provide the ADT service manually in the interim. Regardless of whether a particular service is implemented manually or electronically our guiding principles remain the same. They are:

  • Automate the transfer of data so that you do not have to chase data or engage in data entry work.
  • Do so in a secure manner.
  • Do so in a reliable manner.
  • Alert you to any error conditions that may need correcting.

Security and Reliability

The security and confidentiality of your data is important to us. We encrypt any data that we control that is transferred over the network. All credentials are stored encrypted. All Scope 5 employees and affiliates are bound by our Security Policy for Employees and Affiliates with Access to Customer Data, available upon request.

One of the reasons our clients entrust us to handle ADT on their behalf is for the increased reliability they can expect. While no system, electronic or manual, is 100% reliable, it is our goal to provide 100% reliability. We incorporate thorough data integrity checks, both electronic and manual. 

Error Notification

At times, account credentials expire, meter numbers change and problems may arise in transferring data. As part of our ADT service, you may receive communications, whether by email or in person, alerting you to such problems. Resolving these may rely on your cooperation.

Relationship of ADT and APIs

The terms ADT and API are closely related. ADT (automatic data transfer) is generally implemented using an API (application programming interface). Scope 5 provides APIs of various types, many but not all of which are designed to transfer data.

 

Have more questions? Submit a request

Comments

Powered by Zendesk