SAP ARIBA Extractor

This project is an extractor for the SAP Ariba API. It allows you to extract data from the SAP Ariba platform and save it to a local file in CSV format.

The extractor uses the httpx library to make HTTP requests to the SAP Ariba API and handle the authentication. The data is then parsed and written to a CSV file using the csv library.

The extractor is easy to use and customizable, you can configure the extractor to extract data from different parts of the SAP Ariba API by modifying the endpoints and the parameters of the requests.

The extractor also includes a built-in rate limiter to ensure that the requests are sent at a safe rate and avoid overwhelming the API.

The project also includes a requirements.txt file for you to install the necessary dependencies.

To use the extractor, you will need to have an SAP Ariba API key, which can be obtained from the Ariba developer portal.

The extractor is built in Python, it is compatible with Python 3.10.

This extractor can be useful for businesses that use SAP Ariba to manage their procurement processes, allowing them to extract and analyze data to make better decisions, automate processes, and integrate the data with other systems.

Installation

pip install -r requirements.txt

Before use

To use the tool, you need to provide a YAML configuration file that specifies the constants needed to work. Here's an example:

constants:
    p2p_realm: &p2p_realm '****'
    sourcing_realm: &sourcing_realm '****'
    stage: &stage '****'
    base_url: &base_url '****'
    oauth_url: &oauth_url '****' 

    # Procurement
    operational_reporting_for_procurement_api_key: &operational_reporting_for_procurement_api_key '****'
    operational_reporting_for_procurement_client_id: &operational_reporting_for_procurement_client_id '****'
    operational_reporting_for_procurement_secret: &operational_reporting_for_procurement_secret '****'
    document_approval_api_key: &document_approval_api_key '****'    
    document_approval_client_id: &document_approval_client_id '****'  
    document_approval_secret: &document_approval_secret '****'
    master_data_for_procurement_api_key: &master_data_for_procurement_api_key '****'
    master_data_for_procurement_client_id: &master_data_for_procurement_client_id '****'  
    master_data_for_procurement_secret: &master_data_for_procurement_secret '****'
    supplier_data_with_pagination_api_key: &supplier_data_with_pagination_api_key '****'  
    supplier_data_with_pagination_client_id: &supplier_data_with_pagination_client_id '****'  
    supplier_data_with_pagination_secret: &supplier_data_with_pagination_secret '****'

    # Sourcing
    operational_reporting_for_sourcing_api_key: &operational_reporting_for_sourcing_api_key '****'
    operational_reporting_for_sourcing_client_id: &operational_reporting_for_sourcing_client_id '****'
    operational_reporting_for_sourcing_secret: &operational_reporting_for_sourcing_secret '****'
    sourcing_project_managment_api_key: &sourcing_project_managment_api_key '****'
    sourcing_project_managment_client_id: &sourcing_project_managment_client_id '****'
    sourcing_project_managment_secret: &sourcing_project_managment_secret '****'
    master_data_for_sourcing_api_key: &master_data_for_sourcing_api_key '****'
    master_data_for_sourcing_client_id: &master_data_for_sourcing_client_id '****'
    master_data_for_sourcing_secret: &master_data_for_sourcing_secret '****'
    user: &user '****'
    passwordAdapter: &passwordAdapter '****'  
    remove_old: False
    filenames_as_timestamp: False


api: !include extractor/developer_settings.yaml   

There is also a reference to the developer_settings file. For more info on how to configure that file please refer to Extractor Specifics

CLI options

The extractor can also be used as a command line tool. To use the tool, you need to provide a YAML configuration file that specifies the API endpoint, parameters, headers, and authentication credentials. Here's an example:

python main.py --model_name receipt_updated_range_async --date-from 2023-11-29 --date-to 2023-11-30

• model_name: The name of the model to use for the API request.
• allp2p: This flag can be set instead of model_name. This will extract all P2P models defined in a constant called P2P_MODELS in the constanst.py file.
• allsourcing: similar to the previous flag. This will extract all sourcing models defined in a constant called SOURCING_MODELS in the constanst.py file.
• date-from: The start date to filter the API request.
• date-to: The end date to filter the API request.
• debug: Set this flag to have a more deep log.
• delta: Set this flag to download the selected model from last updated item until today.
• output: This is optional, and it expects a folder name where the extracted files will be created.

The extractor expects a model_name or one of the flags: --allp2p or --allsourcing. If none of these is provided, the extractor will terminate and request one option.

Only a few models can be run with the --delta option due to how the endpoints are called. They are:

• copy_order_updated_range_async
• copy_order_updated_range_sync
• direct_order_updated_range_async
• direct_order_updated_range_sync
• erp_order_updated_range_async
• erp_order_updated_range_sync
• invoice_reconciliation_updated_range_async
• invoice_reconciliation_updated_range_sync
• invoice_updated_range_async
• invoice_updated_range_sync
• receipt_updated_range_async
• receipt_updated_range_sync
• requisition_updated_range_with_history_async
• requisition_updated_range_with_history_sync
• counted_copy_order_updated_range
• counted_direct_order_updated_range
• counted_erp_order_updated_range
• counted_invoice_reconciliation_updated_range
• counted_invoice_updated_range
• counted_receipt_updated_range
• counted_requisition_updated_range_with_history
• sourcing_project_events_sync
• sourcing_project_details_sync
• sourcing_projects_sync

Within the rest of the models, only a few can be filtered by creation date. They are:

• sourcing_project_events_async
• sourcing_project_details_async
• sourcing_projects_history_sync
• sourcing_project_history_async
• sourcing_projects_async

The rest of the models can't be filtered by date, so each extraction will download all information. They are:

• currency_conversion_rate
• company_codes
• countries
• payment_terms
• purchase_groups
• purchase_orgs
• suppliers
• uoms
• users
• vendor_list