I've tried Seek by iNaturalist [1] to identify some plants in real time (the app can also identify fungi and wildlife). It's been a bit hit or miss sometimes. I like that it doesn't require any registration and doesn't collect any data. I'd like to know how this compares (at least for plants).
Myco-nerd here: absolutely never rely on the image recognition apps for fungi ID.
There are fungi in entire different genera that are VERY morphologically similar, and the apps are just not there yet — likely won't be for a few dozen years.
An app can make a lethal mistake much easier than a human.
I’ve used Google Lens as “if it says unsafe, definitely unsafe”. I would not rely on it for “oh yeah, I can eat this” :).
I doubt your “few dozen years” though. Humans are only so good at it themselves. Computing has improved a lot since 1984 (3 dozen years ago), and so I’d wager that by 2050 we can be better than human at “Eat or not?” for fungi. Up for a longbets.org wager? :)
I mean the thing about fungi is that the tops can look the same and you just need to do a spore print to positivity distinguish one from another. There may be some mushrooms which are simply impossible to tell apart by outward appearance. In one of the fast.ai lectures Jeremy shows how to distinguish different breeds of cats. Then he shows how to look at the confusion matrix, and he found one pair of breeds the network really struggled with. It turns out they look really similar to him too, and when he researched further he found they’re simply hard to tell apart. Perhaps with an enormous data set there might be small differences a network could detect, but the confidence might still be low.
And given that mushrooms can kill you, it may simply never be advisable to rely on any photo based identification.
I don’t consider it against the rules of the bet to allow multiple pictures, including the underside and perhaps even “here, smush the mushroom on a piece of paper and take a picture of that”. My question is can a vision-based AI thing outperform humans within another thirty years, not if it can do it via a mechanism that isn’t discriminating.
For all the myco folks here: Do you have a sense of whether or not the multiple hours mentioned is “required” or “just” makes it easier to get a strong signal? (That is, how much is the signal boost due to our inability to see well as humans)
We foragers and amateur mycologists use smell, touch (slimy, dry, etc,) sometimes taste (bitter, acrid,...), habitat (on wood, ground, type of wood, is there a bulb below ground or a root-like structure, time of year, spore prints, sometimes color change due to drops of chemicals (especially on boletes), sometimes even microscopes to view spores, and more.
all of these variables could of course be coded for a good classification algorithm.
It's mostly visual. I would add location and latest weather to any software trying to recognize the mushroom. The mushrooms that people commonly forage for are not that numerous, so the algorithm needs to know about a dozen or two varieties. Chanterelles are easy to tell from images. You can probably have an AI chanterelle identifier coded right now and for most edible mushrooms very soon if not now.
Neat :) I just had a little play with that in our garden it got things like himalayan clematis, forget-me-nots, dafodils and chives right. Will have to try on mushrooms etc!
Sometimes when it didn't seem to give a high accurate match it says 'dicots' for the family.
It said hemp family for my hops, I guess it's hard to get an exact match of hops, unless they're flowering I guess. But impressed it got that, it seems a hard problem!
Edit: it did get hops eventually after trying a different angle :)
I heard about this app on the radio yesterday and I literally just got back inside from using it to identify a plant outside my house. It worked ok. when I went to install it, there were a number of other apps that came up in the app store and I downloaded picturethis which identified the plant instantly whereas inaturalist gave me a list of possible plants to pick from. picturethis has some in subscription stuff going on that I didn't look into tough.
I haven't tried Seek, but I've had better luck with animals than with plants on iNaturalist, personally. It's useful to algorithmically ID the plant family.
Looks great. The app doesn't appear to be open source, just the API docs: https://github.com/plantnet
Given the public funding of the project, wouldn't it make sense for the code to be open to welcome contributions/improvements from the community/users?
iNaturalist is publicly funded, a wide collaboration, and IIRC mainly FOSS and Open Data. They're using machine learning and knowledge of users to identify flora and fauna; I've found it excellent at doing automatic ID at the genus level (but not really species).
Depending on what you are interested in (and where) there will be plenty of enthusiastic users willing to identify your observations. In my experience, birds will always get identified, while insects and plants will only sometimes register interest from others. I think it's because bird watching is a popular hobby, while entomologists tend to be a rarer breed.
There are also at least 2 orders of magnitude more species of insects on Earth than of birds (>1 million compared with ~10,000). An amateur birder can know a significant fraction of all birds, and all of them commonly found in their location, while no entomologist is anywhere close to knowing all insects.
Absolutely, a good point. I would think for the particularly avid birder that you could probably memorize all of them. We certainly have larger vocabularies than that!
Google Lens does this. But not only for plants. And it is quite good. I could identify most of the plants in the park. Cool to know there is some edible stuff in the park like ramson
Were those ramsons flowering? Lily of the valley leaves could easily trick Google Lens, they would be more likely to find in a park, and they are deadly.
I've used this for a while but I'm never really sure about the results. Honestly I had no idea that identifying trees accurately could be so difficult - and I mean as a human with access to photographs and Wikipedia and books and other people.
Yeah, it's wonderful how complicated the natural world is.
We're still discovering stuff every data. Recently it was discovered that one of the most unique looking critters in the world (the mata-mata, a turtle) was actually two species.
In even grander confusion, a friend of mine was working her way through her masters in mycology. She focused on the fungus Phytophthora ramorum. In the end it turned out that the Phytopthera aren't even fungi. They're algae, an entirely separate branch of the eukaryotes. It's like discovering that your cat is a house plant.
Thirty years ago I was walking farmers soybeans fields, making weed maps and then writing prescriptions of what to spray. One feverish afternoon with the temperature hovering in the nineties I had an idea. What if we had a plane flyover, grab images and have a software program identify the weeds and spit out a prescription?
I talked to some professors at Michigan State and other software developers I knew. The consensus was it was a great idea but the technology simply was not there or even remotely close.
Little did I know several years later Monsanto would come out with genetically engineered soybeans that you could simply spray Roundup over. Walking fields, weed maps and prescriptions of chemicals became a thing of the past.
To me it was an excellent example of how technology can blind side you at times. Now of course weeds are becoming resistant to Roundup and prescriptions might make a comeback!
I tried something similar with Google auto ml a while back. Although I'm no ml expert, I think it's quite difficult to identify plants. There are many different parameters like distance, flowering or not or even time of the year. In some cases it can also be quite dangerous to mix up plants, like wild garlic and lilly of the valley ( I hope I translated those right)
I've had this app a while and just used it to identify Eastern Skunk Cabbage growing near me. I didn't know that some plants could be thermogenic.
This app seems to work pretty well. I tried this identification first on a single leaf and only got results for plantains. But using a pic of the whole plant got an accurate ID.
I’ve been trying to determine whether the oak in my front yard is a scarlet oak or a pin oak. The two varieties have very similar leaves.
According to the app rankings, based on acorn, leaf, and flower pics, it is a downy oak. Not perfect, but I am impressed that it picked up that it was an oak at all.
What about mushrooms? What about slime molds? What about insects? Google Lens does all of the above as well as plants and flowers and fruit and shoes. I think PlantNet looks nice, and I appreciate the web-only version! But Lens is built into my camera, it's hard to beat.
What I think would be useful is an app that could diagnose problems with plants. Some get yellow when dry, some get yellow when too wet. You could tell it the species and it could compare it to training data of sick plants of that species.
I've just tried PlantNet for the first time. In the one case I tried it on, it seemed better than PlantSnap, which I installed a couple of weeks ago. There's a flower I've been wondering about for years. It looks very like a Forget-me-not, which grows in a number of places nearby and flowers at the same time, but has wider leaves. After quite a few attempts with PlantSnap, it came up with Chinese Hounds Tongue Forget-me-not[0], which sounded plausible. However, after a much smaller number of attempts with PlantNet, it suggested Green Alkanet[1], which I'm starting to think is more likely, e.g. due it flowering earlier and being more weed like. Not an expert, and I know this is only one test case, but based on this I'm favouring PlantNet. Looks like there will be fewer notifications too.
I can already see how future ML platforms going to gain a lot from crowdsourced data.
Imagine a platform that can publish an app for classification of any particular use case, and the dataset is contributed and vetted by the community of users
For example:
1. App for identifying animals ( like inaturalist )
2. App for identifying a car/bike model versions
3. App for identifying languages and translate
4. App for identifying different kinds of dogs
All these apps need to do is come up with a CAPTCHA like implementation. If you hated "Find all crosswalks", just wait for "Find all Lilium longiflorum".
Agree. They act like they're doing it for public benefit and connected with academia but without being clear if they're a commercial offshoot or charitable or something else. They are accepting donations in a way that implies they're charitable but isn't backed up by clear details (although maybe it's buried somewhere in the site?)
Whether they're planning to be open source with their data would be good to clarify too.
Is there a good plant ID app that's not based on my camera? I was thinking something that will ask me increasingly specific questions. "Where are you?" "What kind of bark does is have?" "How large are the leaves?" Etc
I posted it in a bit of a hurry, and couldn't think of a description that didn't sound like shilling the app. Looks like dang (or sctb?) has put in a perfect descriptor!
[1]: https://www.inaturalist.org/pages/seek_app