Apify

Crawlee for Python is officially live! 🥳 After the success of Crawlee for JavaScript/TypeScript, our community made it clear that they'd love to use it in the go-to language for web scraping: Python 🐍 We've worked hard over the last couple of months to bring this project to life 🚀 Now, we're excited to share the beta version of the repository with you, which is already open for early adopters 🤩 Check out and support the best web scraping and automation library on ProductHunt 👇 🔗 https://apify.it/3zJMzP9

Recently, I had the opportunity to speak at the WebExpo Conference about feeding LLMs with data extracted from the web. Finally, there’s a video recording. The talk covers the importance of web data for training new AI models as well as for RAG—the ultimate killer application of LLMs. However, reliably collecting web data at scale is increasingly challenging due to issues like blocking, dynamic content rendering, and the sheer volume of data. In the talk, I show how to set up an efficient web data extraction pipeline using Apify, clean the HTML to circumvent the “garbage in, garbage out” problem, and demonstrate how to build a new GPT with the cleaned data. Enjoy! https://lnkd.in/eTuwxb8V

Want to get creative and generate a scrolling GIF from any website? 🎬 Actors on Apify Store offer more than just web scraping tools. For instance, the GIF Scroll Animation tool captures multiple screenshots of a website and then turns them into an animated GIF file 🤩 You can even customize it to click buttons, visit other websites, or hide cookie banners! 🍪 Watch our new video tutorial to learn how to use this versatile tool 👇 🔗 https://apify.it/3Wy6Dx5

You can now share your Actor runs 🚀 Whether you need a dev to look at a run, share a dataset with colleagues, or show off the number of results to your friends, there's now a dedicated button for it 🤩 The option becomes available as soon as you start your Actor, so anyone can watch the run without needing to be signed in to Apify 🧐 Try it yourself on Apify Console👇 🔗 https://apify.it/4cU7jCw

We hit the waves for our July all-hands meeting 🏄 Our Office Manager Johana Pogranová organized an evening like no other - directly on the Vltava river, where we had a buffet grill, beach sports, and a beautiful view of Prague 🏰 Even the crazy weather (ranging from stifling summer air to a thunderstorm) didn't throw us off, and we had a fun evening, welcoming lots of new Apifiers this month! 🍻 Thanks to everyone who helped, and we can't wait to see where we'll go next! 🤓

“It always seems impossible until it’s done.” Last week I went to Prague at Apify HQ for the launch of Crawlee for Python and EuroPython 2024. Last few weeks were very busy for the team preparing everything but it was totally worth it. Few achievements from our launch last week: - 2500 stars on Crawlee Python GitHub repo. ✨ (https://lnkd.in/gaxisHbc) - Thousands of new visitors 🎖️ - Trending for straight 5 days on main GitHub trending page. 🚀 - For more than 24 hours on HackerNews main page. 🧑🏻💻 - Top Post on Reddit Python subreddit. 🐍 - Many early adopters and a lot of good feedback from the developer community. ♻️ - Team managed to get a booth at EuroPython and we did a lot of demos there and had amazing talks with devs (Special thanks to Michelle Macias Mendez) 🎊 - And gave a lightning talk at this amazing conference about Crawlee for Python. 🗣️ Very happy with the response from Python developer community and much excited to work further to make this library the BEST web scraping and automation library in the world. 🕸️ To many more such projects. 🥂

Want to get data from Indeed in a structured format? 💼 Scraping Indeed is as easy as if you were browsing the website itself! All you need to do is input the job position and location 📍 You'll then have your results in minutes 🚀 Watch our new video tutorial to see how you can scrape job listings with Indeed Scraper on Apify Store 👇 🔗 https://apify.it/3WtcN1v

How does climate change affect riverine communities living along the Amazon River? 🌎 An article in the Communications Earth & Environment journal on Nature finds settlements at risk of isolation during low-water periods and sees the impact reported in digital news outlets 🧑💻 Apify was used for the data collection of articles documenting these events that serve as evidence of how climate change disrupts water transport and limits access to essential goods and basic services 🤝 This article is yet another example of how automated data extraction can help us make better use of the amount of information available online - not just in business, but also in academic research🔬 Read the full article on Nature 👇 🔗 https://apify.it/3LvZU0i

The new Apify Store offers hand-picked Actor collections to showcase scrapers that are worth your attention 📣 Check out some of the Actors featured in 'Actors from community experts', a spotlight on exceptional and user-recommended tools you shouldn't miss 💪 Currently, the collection includes: • eBay Scraper • Reddit Scraper • Work Scraper • API / JSON scraper • Yellow Pages • Linkedin company scraper • Apartments com Scraper • G2 Explorer • Email ✉️ & Phone ☎️ Extractor Keep in mind that the Actors within these collections are dynamic and will be updated periodically 🚀 So stay on the lookout and browse the collection via the link below 👇 🔗 https://lnkd.in/eizwChxw

You might not have noticed, but Clay just added 1,514 new data sources! This upgrade is a game-changer for #salesteams, offering access to a wealth of data including: ✅ Amazon product information ✅ LinkedIn job postings ✅ Web traffic insights ✅ Hiring trends from Indeed ✅ Funding round data ✅ Instagram account details ✅ Local business data from Yelp and Yellow Pages And that's just the beginning. All this is possible thanks to #webscraping that leverages residential proxies, providing access to valuable public data. The new Apify integration in Clay brings you a marketplace of top-tier scraping tools, perfect for those custom data needs without the hassle of doing it yourself. Are you building the next big public-data SaaS? I can help you unlock access to over 85 million NetNut.io's premium residential proxies. Interested? Let me know! 🚀✨