HANA Cloud Integration (HCI)-Data Services and ODP Extractors

SAP has lots of Standard Extractors for Business Content tailored for BW, and these contain all the common business application logic & can identify data changes or delta and these SAP Extractors are used to Load Data into BW , and these Loads are done without using Data services.

 

SAP Netweaver 7 Enhancement Pack 3  and above provides technical framework for consumption of SAP Business Suite Operational Analytics by (Extract Transform Load) ETL Tools, termed as Operational Data Provisioning (ODP) . It is one uniform interface (API) for consumption by analytics frontends like  BEx or BOBJ Explorer.

What it  means is that  Like BW  which’s been using SAP Extractors for business content, Non BW customers can also get same access to SAP Business Content without requiring to go through BW application.

Using ODP Technology framework business content is also available for consumption for HANA Cloud Integration (HCI)  Data Services tool , ETL tools and analytics frontends including mobile.

HCI Data services has many ways to connect to SAP Business Suite , most common is using ABAP Data flows , in which an ABAP Program is executed on the ERP server i.e. It Reads Tables via ABAP,  and with availability of ODP APIs HCI Data services can  also consume data from SAP Extractors , similar to reading from SAP Tables.

Reading Business content from these SAP Extractors are reasonably useful & good source for HCI Data services, as apart from getting consistent application logic , transforms and  aggregates which BW gets,  the dataflows can use the same full & delta loads features BW uses, that too without using BW.

Since ABAP program can read business content from SAP Extractors using ODP APIs i.e. these are nothing but ABAP based Function Modules to Read Data records Queue Table with ability to specify  data filters and projections criterion as Function module import parameters. It opens up possibilities to extend the SAP solution tremendously. This point needs separate discussion, perhaps i will write about it.

Extractors can be selected from SAP / ECC DataStore by using  Import Object by Name Option – or or use Import Objects by application area option.

One can browse and import extractors (those which are released by SAP) just like SAP Tables. Note not all  available SAP Extractors for BW have been released for ODP and not all extractors support delta. As shown in the image below , Choose Type as Extractors from the drop down menu ( default is Table). Note the option for Extraction mode as query corresponds to full data load  ( from the delta Queue table) and Change Data Capture (CDC) corresponds to Delta load ( from the delta table – corresponding BW extractor uses) .

Note ODP Technology framework supports multiple subscribers to replicate data and in parallel same SAP Extractors can be used as BW Data source connected via BW delta queues ( Transaction RSA7  for BW delta queue monitoring). Each subscriber can independently  request a data snapshot (full or one off request)  or request data changes (delta request), as last successful run data pointers are seprately maintained  for each subscriber in the ECC ABAP based system. Transaction ODQMON can be used to monitoring the operational delta queues subscriptions.

Once Imported , extractor Can be Dragged & Dropped into Data flow , similar to dragging  SAP Tables and mapping the fields in transforms and query joins.

 

 

Extractor Options that need consideration in the design are Initial Load option of Yes or No . Initial Load value yes indicates to extractor to read all its data and not just delta and if set to No , to read delta from the Extract from Date time value, specified by the global variable like $G_SDATE chosen in this option field.

This date value is used to associate an Extractor run with a date. Though not needed for the initial load , helps in building delta loads.

After Initial Load of data ODP API not only returns the data, & ‘initializes” the delta queue so that next delta will return all rows which for changed in the meantime, and the extractor date-time is set to $G_SDATE variable value of whenever last time successful data load happened.

Most cases will have two data flows one with Initial load for bulk data load and delta loads. Package size control settings for data transfer in MB using the ODP framework.  Word of caution,  logistics extractor are dependent on setup tables, data from which is commonly deleted after use of BW for initial or full load & subsequent changes or delta records are updated to delta queue . In my observations, Initial load using extractor’s limited to what’s available in delta queue unless setup tables are  filled again.