Usage Note 66065: Teradata database space requirements for the SAS® Embedded Process for parallel data loads and saves
In order to do parallel loads of Teradata tables into CAS and parallel saves of CAS tables into Teradata, additional space is required in Teradata for use by the SAS Embedded Process.
- During the data transfer process, a volatile temporary table is created to hold a small SAS DS2 program that runs in the SAS Embedded Process. This program performs any of the data type conversions that are required to map the Teradata data types to SAS® Viya® (CAS engine) data types, including numeric conversions and transcoding character data to UTF-8. In SAS Viya 3.5, if you find that the table is not being created as a volatile temporary table, see SAS Note 63630.
- The space for the volatile temporary table that is used for this purpose comes from the Teradata user’s spool space. For permanent tables, the spaces come from the perm space.
- The DS2 program is not large and is stored as a Teradata BLOB. Teradata stores BLOBs in a subtable on the same AMP with the Teradata base row. So, by their very nature, BLOB contents for a given row affect the storage on a single AMP. You need to consider this behavior when computing the amount of space that needs to be allocated to accommodate SAS Embedded Process BLOBs. Teradata space limits are set globally. However, each AMP gets the same portion of the global limit, so the space limit for the SAS DS2 BLOB must account for the number of AMPs. Here is an example calculation:
Size of BLOB for Data Connect Accelerator for Teradata <= 100KB
Number of AMPs = 216
Space limit to accommodate BLOB = 100KB * 216 = 22MB spool space (volatile) or perm space (permanent)
- When the SAS Embedded Process runs on the Teradata system, it logs any SAS processing messages to a global temporary trace table. After the SAS Embedded Process completes its work, the Data Connect Accelerator on SAS Viya reads this global temporary trace table and transfers any messages related to the running of the SAS Embedded Process to the SAS log. Global temporary trace tables require Teradata temporary space. This is a requirement for proper function of the Data Connect Accelerator for Teradata. Other SAS Embedded Process In-Database Accelerators (SAS® Scoring Accelerator for Teradata, SAS® Data Quality Accelerator for Teradata, and SAS® 9.4 DS2 Code Accelerator for Teradata) have the same temporary space requirement.
- SAS Embedded Process messages are usually just a few lines of text, generated on all AMPs or on a single AMP. The temporary space limit that is needed is also based on the number of AMPs:
SAS EP messages <= 1KB
Number of AMPs = 216
Space limit to accommodate global temp trace table messages = 1KB * 216 = 216KB
- For loads, the user also needs enough spool space to materialize the data that is being transferred to SAS Viya. This requirement is not different from what would be required when the user performs a “SELECT *” from the table or view.
Operating System and Release Information
SAS System | SAS Data Connect Accelerator for Teradata | Linux for x64 | V.03.05 | | Viya | |
*
For software releases that are not yet generally available, the Fixed
Release is the software release in which the problem is planned to be
fixed.
Date Modified: | 2020-07-13 14:12:26 |
Date Created: | 2020-06-01 16:29:35 |