Building an automated content curation workflow with Integromat, Airtable and Mercury
Here at FTN we send a daily newsletter to hundreds of subscribers in which we highlight not only our own content, but also interesting articles and videos from around the web. they serve as a constant reminder that we’re living in world that becomes smarter, better and healthier everyday, despite all odds. Members of our team and, occasionally, newsletter subscribers send our editorial staff readworthy links.
By the way: You’re invited to subscribe to our newsletter.Not only will you see our system in action, you will also receive some of the best reading suggestions on the web. If you’re a happy subscriber already, why not recommend it to your friends?
When we invest time in creating the newsletter we want to focus on reading the content, writing teasers, and crafting headlines. Therefore we want to cut down on copy/paste operations as much as possible.
We had these goals when we set up this process:
- Simplify link submission as much as possible. The process follows user behavior, not the other way around.
- Capture relevant data when someone submits a link so we can attribute it properly (if the submitter wishes so).
- Capture metadata such as title, author and teaser for the link to to streamline the review process and have a consistent presentation.
We have experimented with different services and workflows, among them Zapier, Google Spreadsheets, Pocket, and Feedly. But now we settled with three online services to fulfill these goals: Integromat, Mercury and Airtable. Good old eMail is also in the mix.
Our Tools
Integromat
Integromat is an online automation service like IFTTT or Zapier. It connects to your accounts on cloud services and processes your data for use in different services. It can also move data through different services for advanced needs. In our scenario, it serves as the “glue” between Email, Mercury, and Airtable. We’re gladly paying customers and use it to simplify our editorial work in all aspects.
Mercury
Mercury by Postlight Labs is a free service that parses webpages and extracts articles and metadata for uncluttered reading. It then returns them as data structured in JSON. You can then use it in all kinds of services and Apps (like Integromat). We use it to extract metadata, such as author, title and publication date from our link submissions.
Airtable
Airtable is like a spreadsheet/database hybrid on steroids. It’s easy to use and very powerful at the same time. You can structure and link your datasets with each other. We use it as a hybrid system for Contact Management, Content Planning and Operations Monitoring. This gives us unprecedented insights into our work and allows us to get better every day. For this example, we store all the information on the links and add our own titles and teasers. We pay for Airtable and we don’t want miss it.
Journey of a link
OVERVIEW OF OUR WORKFLOW
When we discover a link that we think could be included in our newsletter we send it to the group email address collection@ftn.media. We have the convention to include a meaningful subject line and send only the url in the body. Our GSuite spam filters keep the dirt out. One of the recipient addresses is a special Integromat address where the email is immediately processed.
EXTRACTING THE LINK
Then we run an operation on Integromat that extracts the url from the email text. If this fails (which is rare), we send an answer explaining the error and a slack notification to a special Slack channel to investigate.
And now comes the part where Integromat really shines. It’s so versatile that it can connect to Mercury’s API without being pre-configured (which we couldn’t do with IFTTT or Zapier).
MERCURY IN ACTION
So Integromat sends the extracted link to Mercury. Mercury responds with the extracted article and metadata in JSON format.
CONVERTING JSON
Now Integromat processes the response by extracting the data from the JSON structure and it’s ready for further use.
PUTTING IT INTO AIRTABLE
In the last automated step Integromat creates an Airtable entry for the link. It contains the URL, the title, the Author, the Source, the publication date, the submission date, and the email-dress from which it was submitted.
Now we have everything we need to know in one place. And we didn’t need to copy-paste a single word of text.
Manual Work
The time we just saved can be used for better editorial work. We did a quick test and found out that we would spend almost two minutes manually copying all this information. Since we’re processing 20 – 30 links daily we already saved two to three hours per week.
We use this time to carefully read the links, sort them, correct errors and add bespoke titles and teasers.
EVERYTHING IS READY FOR POLISHING IN AIRTABLE
Once we’re ready, we have a field in Airtable that combines titles, teasers and URLs into HTML code.
We can now copy the contents of this field for all selected fields at once and paste them into Mailchimp where we finalize and send our newsletter (did you consider subscribing?).
Roadmap
We’re very happy with our workflow and we want to expand it in the future.Once the respective features are supported, we will consider the following features.
- Generating and sending the newsletter from Airtable without any manual steps in Mailchimp
- Crossposting the links to a tumblelog and our twitter feed.
- Pinging authors and publications on twitter when we include their links.
Before you go: You’re invited to subscribe to our newsletter.Not only will you see our system in action, you will also receive some of the best reading suggestions on the web. If you’re a happy subscriber already, why not recommend it to your friends?
