Automation Challenge: Ensuring Complete Data Scraping for Competitors
Hey everyone, I’m working on an automation to enrich competitor data for a Google Sheet. The flow looks like this:
- 1.
Get Competitors (from Google Sheets – company names and social media links like Instagram, YouTube, TikTok, LinkedIn).
- 2.
Enter Loop to process each competitor one by one.
- 3.
Search Perplexity for more info.
- 4.
Scrape LinkedIn profiles via URL.
- 5.
Scrape YouTube, Instagram, TikTok profiles via their respective scrapers.
- 6.
Update the row in Google Sheets with all gathered data.
Here’s the issue: The loop seems to exit before all scrapers (YouTube, Instagram, TikTok) are finished. I want the loop to continue only after all scrapers have returned their data for a competitor. Does anyone have an idea how I can make sure all scraping steps are completed before exiting the loop and moving on to the next competitor? Thanks a lot for your help! 🙏
