It's fun to wake up to find a project that I started on the top of HN! These days, I'm no longer very involved with the day-to-day of the project.
Now that it's no longer a young project, here are some musings about Huginn and responses to people's comments in this thread, in no particular order.
I've found that Huginn excels as a scheduled web scraper with lightweight filtering. That's what I use it for. On the other hand, while you can write custom code in it, Huginn is pretty poor at implementing any sort of complex logic, and is even worse at bidirectional syncing between systems, which is something people often want it to do, but for which it wasn't designed.
If IFTTT or Zapier meet your needs, awesome! No need to run and monitor your own service. I personally choose to run Huginn on my own hardware in large part so that I'm comfortable giving it website cookies and passwords.
Some examples of what I use Huginn for these days:
- Watching Twitter in realtime for high standard deviation spikes in certain keywords, such as "san francisco emergency" or "san francisco tsunami warning", which then sends me a push notification, or "huginn open source", that goes to a digest email (and I imagine will trigger because of this thread).
- Watching Twitter for rare terms and sending me a digest of all tweets that match them. Also sending me all tweets from a few Twitter users who post rarely, but that I don't want to miss.
- Scraping a number of rarely updated blogs that don't have email newsletters and emailing me when they change. Some use RSS, most are just simple HTML scraping.
- Pulling deals from the frontpage and forums of slickdeals and craigslist and filtering them for certain keywords.
- Sending an early morning email if it's going to rain today.
- Watching ebay for some rare items.
- Sending my wife and me an email on Saturday morning with local yardsales from craigslist.
- Watching the HN and producthunt front pages for certain keywords.
Basically, anytime I find myself checking a website more then a few times, I spend 20min making a Huginn Agent to do it for me.
I think one reason Huginn has worked well for me is that I don't try to make it do too much. I use it for scraping data and gentle filtering, and that's about it. It's been super helpful for alerting me to interesting content for The Orbital Index, my current project, a weekly space-industry newsletter. (Last issue: http://orbitalindex.com/archive/2019-12-10-Issue-42/)
I'm excited to check this out, but I wanted to congratulate you on a truly excellent project name. Having spent many, many hours in naming struggles, I truly appreciate the perfection.
And for those unfamiliar, the (also amazingly named) historian Snori Sturluson explains: "Two ravens sit on his (Odin’s) shoulders and whisper all the news which they see and hear into his ear; they are called Huginn and Muninn. He sends them out in the morning to fly around the whole world, and by breakfast they are back again. Thus, he finds out many new things and this is why he is called ‘raven-god’ (hrafnaguð)." [1]
There is a lovely two player board game called Odin's Ravens in which players acting as Muninn and Huginn race against each other by playing landscape cards. [1]
I wasn't aware of the full details of the lore so thank you for bringing that up!
I thought what a weird coincidence that they'd choose this very similar weird word ... but now that you point out it's the name of one of Odin's ravens, I guess it's not quite as uncommon of a word.
Little known fact: this project is used widely by journalists who can't code (at The New York Times, among others) to do a variety of tasks, like eg. monitoring web pages like Trump's policy position, scraping press releases or filtering out very specific news alerts.
Great to see you at the top of HN today tectonic! I'm at Pivotal in Sydney Australia now. Thanks again for the impromptu interview all of those years ago!
Now that it's no longer a young project, here are some musings about Huginn and responses to people's comments in this thread, in no particular order.
I've found that Huginn excels as a scheduled web scraper with lightweight filtering. That's what I use it for. On the other hand, while you can write custom code in it, Huginn is pretty poor at implementing any sort of complex logic, and is even worse at bidirectional syncing between systems, which is something people often want it to do, but for which it wasn't designed.
If IFTTT or Zapier meet your needs, awesome! No need to run and monitor your own service. I personally choose to run Huginn on my own hardware in large part so that I'm comfortable giving it website cookies and passwords.
Some examples of what I use Huginn for these days:
- Watching Twitter in realtime for high standard deviation spikes in certain keywords, such as "san francisco emergency" or "san francisco tsunami warning", which then sends me a push notification, or "huginn open source", that goes to a digest email (and I imagine will trigger because of this thread).
- Watching Twitter for rare terms and sending me a digest of all tweets that match them. Also sending me all tweets from a few Twitter users who post rarely, but that I don't want to miss.
- Scraping a number of rarely updated blogs that don't have email newsletters and emailing me when they change. Some use RSS, most are just simple HTML scraping.
- Pulling deals from the frontpage and forums of slickdeals and craigslist and filtering them for certain keywords.
- Sending an early morning email if it's going to rain today.
- Watching ebay for some rare items.
- Sending my wife and me an email on Saturday morning with local yardsales from craigslist.
- Watching the HN and producthunt front pages for certain keywords.
Basically, anytime I find myself checking a website more then a few times, I spend 20min making a Huginn Agent to do it for me.
I think one reason Huginn has worked well for me is that I don't try to make it do too much. I use it for scraping data and gentle filtering, and that's about it. It's been super helpful for alerting me to interesting content for The Orbital Index, my current project, a weekly space-industry newsletter. (Last issue: http://orbitalindex.com/archive/2019-12-10-Issue-42/)