Benefits of automating ingest vs manual
- Reduces chance of user error.
- Data files are retained for a period which can help when troubleshooting data and ingest issues.
The data ingest configuration must be tested and ready and a Box folder linked to the app.
1. Set up Box folder to match standard structure and folder names
2. Link to app
3. Create Automation
4. Select data ingest configuration, note file pattern, and select time zone.
- The * symbol is a wildcard.
- The filename must be unique to be picked up by the automation.
- Clients often replace the wildcard with a date and time stamp to provide uniqueness.
5. Set up Integromat cleanup of Box folder. The folder can only account for 99 files at a given time.
Time cadence - special considerations to timing of when we pickup or ingest files
- Work with client to determine business requirements for cadence and scheduling. Best practice is to set up routine ingests at off hours to preserve ingest performance.