I've been learning Japanese for just over a year now, and its a community that might have the highest concentration of amazing free learning apps with sentence mining capabilities, many of which do things very similarly to what you're building. Have you tried some of these?
asbplayer
Manatan
Yomine
Anki Miner
AnimeCards
Nagare
These are just for video, there are other similar solutions for games, manga and ebooks, and some of them are all in one
I've only heard of asbplayer and Anki miner before, the others I haven't. It's a great space tbh, people are building all the time. For me specifically it's about trying to serve all languages to the best of my ability, alot of the ones I've come across only focus on Japanese or a small selection of languages. Also I think there's scope to improve UX and reducing friction for non tech savvy people
This is my main issue with tools like these.
Automated subtitles are inherently inaccurate, because much of language depends on context which you cannot get just from a brief snippet of something. This wouldn't be much of an issue if the tool was targeted at people dedicated to creating subtitles, but if you're trying to learn a new language, how are you supposed to know when the output is wrong or not?
I build the exact same system, but for (mandarin, simplified) Chinese which i'm learning.
Wanted to create such a system for a while, but it's a lot of hooking up the correct libs and making a UI etc. never found the energy.
Thanks to 'ai' I was able to do this finally, think more people needed this push :-)
It's web-based and can be self-hosted in a docker.
Nice! Yeah I think the functionality is never very complicated, I'm more so interested the UX and trying to add features that reduce friction and slowly build an app that can be stable and relied upon.
Thanks! There is a whole like subsection of language learning called immersion based learning. Fictional sources can be OK just depends on how specialised and unique the vocabulary is.
Yeah that is my suspicion too, I am considering adding an option to transcribe system audio too so it could hook into whatever video or audio is playing on the computer. That would be quite a big change so would like feedback first! hahah
I love having auto-generated subtitles available, but I am still skeptical that we can call them "accurate" yet. For many languages the models are quite weak, and even for English and German on YouTube I see errors all the time.
Even for English (which surely has the best support) anything outside of urban American accents is still incredibly weak - you can pretty much guarantee very noticeable mistakes on even highly intelligible mainstream urban British/Irish/Aussie/Kiwi/SA accents. Stronger regional / rural accents & dialects usually render generated subtitles entirely useless.
Definitely a domain a million miles from being solved if even English is this bad.
Yeah I agree the Youtube ones are not very good, I am not sure what they have implemented behind the scenes for it. However this uses the Whisper Large model which is quite a bit better than the Youtube auto generated ones. Only catch is you need to have the video on disk and wait a bit longer lol
Youtube is very bad, but I have honestly not yet seen anyone implement this well on any platform, which indicates to me that it's not a solved problem. Very open to being proven wrong here though - do you have examples of Whisper Large in use in a large consumable sample?
Yeah that is a fair point, it can be replicated but not many people want to replicate an app and would rather pay for it. The functionality is not hard to replicate but the UX side of things and general support and improvements over time is what people have paid for. There is a lifetime option also - yearly was about offering choices.
Same goes for Dropbox - you can already build such a system yourself quite trivially by getting an FTP account, mounting it locally with curlftpfs, and then using SVN or CSV on the mounted filesystem.
asbplayer Manatan Yomine Anki Miner AnimeCards Nagare
These are just for video, there are other similar solutions for games, manga and ebooks, and some of them are all in one
I've been learning Japanese as well and the content at my level is boring while the content I'm interested in is a wall.
Not seeing a reason to switch atm tbh
This is my main issue with tools like these. Automated subtitles are inherently inaccurate, because much of language depends on context which you cannot get just from a brief snippet of something. This wouldn't be much of an issue if the tool was targeted at people dedicated to creating subtitles, but if you're trying to learn a new language, how are you supposed to know when the output is wrong or not?
While I'm not so sure about learning a language with content from fictional sources, that should at least make it more relevant for the learners.
And you can use it on podcasts too, where people talk normally.
The biggest hurdle is probably that users need to get hold of the media so they can use it as input.
Yeah that is my suspicion too, I am considering adding an option to transcribe system audio too so it could hook into whatever video or audio is playing on the computer. That would be quite a big change so would like feedback first! hahah
Definitely a domain a million miles from being solved if even English is this bad.
I'm confused why is this a yearly subscription or paid at all?
This tool might as well be open source to be brutally honest.