Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I have been running huginn on my home server for a while. I've mainly used it to filter RSS feeds and then generate new feeds with the filtered items. Another use-case for me is ingesting webcomic RSS feeds (or scrape a page) and post the comics to a private Telegram channel. Once I also had an agent scraping a page and notifying me if something changed (realestate listing).

I have tried a couple alternatives, e.g. node-red but none really worked the way I wanted them to for these cases. huginn is incredibly flexible and (at least for me) the mental model of it's workflow makes a lot of sense. Sadly more and more pages want you to go through their app/site and make it a bit difficult to work with, e.g. getting content from an instagram account.

One thing I have not figured out about huginn and which all of these automation tools seem to lack are loops. E.g I have page an agent scrapes, from which I want to output the src of an image tag but I also want to check if a certain condition on the page matches (e.g. a "next page" button exists) and then firstly output the found src but then also re-invoke the agent with a new input element. So it would scrape the next page and so on until it does not find the button anymore.



> One thing I have not figured out about huginn and which all of these automation tools seem to lack are loops.

I do some very complex stuff using the "Shell Command Agent". You might want to look into using that if you haven't already.

You can also create your own agent gem https://github.com/huginn/huginn_agent


You could perhaps use a tool like scrapy in combination with huginn. Scrapy has very simple patterns for doing that kind of logic.


Yeah I just use the Javascript agent for v complex stuff like this.


Oh, how could I have missed the JS Agent... thanks a lot for the hint! But it looks like it's not node, so no npm modules and other node APIs.


That's correct. You could shell-out to node with the Shell Command Agent though.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: