I'm pretty leery of making the "access key" and "secret key" so public (like typing them into a web page, or setting them in environment variables). Of course it adds significant friction to set up an IAM identity for every user, and "low friction" is one of the key requirements here.
ryanianian 19 days ago [-]
A "correct" implementation would give you a temporary IAM role or something (STS) based on a JWT or other authn mechanism.
This is not that difficult if you're already invested in an identity ecosystem, but a right pain without something to bootstrap it.
On the plus side, AWS creds can be made to be temporary and limited in scope to just the nouns/verbs required. Creating and vending those tokens is an exercise for the reader.
420official 19 days ago [-]
It really isn't that challenging to get going with JWT auth in AWS. Gitlab has pretty good documentation for how to use Gitlab ID tokens to assume roles that includes everything other than how to generate a JWT here: https://docs.gitlab.com/ee/ci/cloud_services/aws/
And of course generating OIDC PKI JWTs is pretty easy and well documented elsewhere.
The harder parts in my mind are:
- Updating this OSS project to serve a JWK from OIDC .well-known
- Convincing people that this method of authn is safe and that those keys are securely stored
rohitghumare 19 days ago [-]
I completely Agree on this point. I have this in mind for implementation. For now, I'm focusing on bringing more cloud providers.
jatins 19 days ago [-]
might be fine for internal company use but pasting access/secret key on a third party website will get you a call from security...or worse, won't
lizzas 19 days ago [-]
My initial though. At least create an IAM user per file :-). Maybe that defeats the convenience.
Cloning and installing is also an option.
rohitghumare 19 days ago [-]
I can add this option with just a few changes in a code, but how can it be easy to use for any team?
lizzas 19 days ago [-]
I am not an IAM expert but maybe the app should have an admin login that sets the IAM user with full permissions on any s3 bucket(s) needed for the app to work.
There should be instructions on how to set that IAM user up (dont make it the root! It just needs full access to a single bucket ideally).
bhawks 19 days ago [-]
Magic Wormhole gives you secure file transmission for free.
I agree with other posters that long lived non-autorotating IAM/S3 secret keys are not a good idea. A common alternate approach is presigned URLs. And not just on S3.
Google Cloud, Backblaze, Digital Ocean, Cloudflare, Azure all have this presigned URLs functionality too (I checked for the degree of lockin before I started using S3's presigned URLs in a set of bulk-data APIs at one place I've worked.)
Also, presigned URLs can be used not just for downloading files with a temporary URL but can also be created for uploading files with a temporary URL.
ryanianian 19 days ago [-]
Magic wormhole is great for live, peer-to-peer transfers.
But it is not great if you want to distribute a file multiple times, asynchronously, or with other functionality gained from centralized storage. This is where people typically use email, dropbox, or perhaps the tool from TFA.
nijave 19 days ago [-]
I think you can seed torrents via HTTP but you'd want to add a layer of file encryption in case someone discovers your torrent
Non-free Resilio Sync basically automated this
up2isomorphism 19 days ago [-]
You guys waste too much time building “open” stuff on S3.
nobodywillobsrv 19 days ago [-]
This is what I was wondering ... why start with s3? Why not "simply" build on IPFS or something like that as a start. Even a shared syncthing key would be MVP.
rohitghumare 19 days ago [-]
Quick Update: I didn't expect so many people to show interest in this project. As a few suggested, I plan to add more storage solutions in the next few days.
Testing Other Cloud Providers:
Backblaze B2 ($5/TB/month)
Wasabi ($6/TB/month)
Google Cloud Storage ($20/TB/month) - I am ready to deploy to production.
DigitalOcean Spaces ($5/TB/month)
Cloudflare R2 ($15/TB/month)
Hetzner Storage ($3/TB/month)
scarface_74 19 days ago [-]
This seems like such a horrible, insecure idea that would never pass muster at any company I’ve ever worked at
woranl 19 days ago [-]
If you have a dedicated bucket just for this and the authentication is just for that bucket, then is it still considered insecure?
scarface_74 19 days ago [-]
Even having long lived access keys and secret keys anywhere is insecure.
There really isn’t a need for them. If you are running the application on any AWS compute - EC2, Lambda, ECS, EKS, etc, there is an IAM role attached to the VM that gives code permission to run.
On the client facing side, it should be connected to your Orgs SSO solution so when a person leaves the company, you deactivate the user in one place.
Besides, I can’t think of any organization of even 2 people that isn’t already using Office365/OneDrive or Google/GSuite with plenty of shared storage. The cost per seat for either is $6 - $25 per user.
perching_aix 19 days ago [-]
I guess this is for smaller organizations with no MS365 subscription (and thus access to SharePoint)?
justmarc 19 days ago [-]
Does MS365 cover all potential use cases, needs and scenarios?
perching_aix 19 days ago [-]
All in the world? I suppose I haven't tried it for recipes, but I guess Copilot could help with that too.
As far as sharing files goes though, yes. I mean, it allows you to... share files, and do so in a controlled manner. Even edit them in-app as long as they're of a supported format.
iJohnDoe 19 days ago [-]
I obviously wouldn’t put my key into a third-party site. However, is there any concern with the self-hosted solution? Looks pretty cool and wouldn’t mind using it. Just make a key specifically for this?
rohitghumare 18 days ago [-]
Yes, feel free to try open-source version available on Github.
Just published a new release to support multiple cloud storage options.
Thanks
rohitghumare 18 days ago [-]
New Release is out now
> We support 7 cloud providers now.
> Beta storage providers are still being tested.
Hi! I’m the author of another open-source project in the same space that I’ve been working on for a while: Filestash [1]
Were you familiar with Filestash before starting this? If so, was there something specific you felt was missing that inspired you to create your own solution? Would love to hear your thoughts.
Your reply comes off as arrogant and hijacking someone's announcement like this is tacky. "Were you aware I had already done this better. Why would you do something I already did? Did I mention I already did this?"
rohitghumare 19 days ago [-]
Never came across this, Interesting project. Thanks for sharing
19 days ago [-]
lomkju 19 days ago [-]
Nice!
Could you tell me why Google Drive didn’t work for you?
Google Drive seems to cheaper and has better UX than S3
- $8 for 2 TB (India)
- Supports file versioning
- 750GB bandwidth per day
folmar 19 days ago [-]
If you do any less typical files Google Drive will balk at you at all times, at least for the other users who use the browser UI.
What works bad:
* folder with lots of medium size files
* large ZIP
* video bigger than a few tens of megabytes
7bit 20 days ago [-]
Why should I use your solution and not Next cloud, for example?
rohitghumare 20 days ago [-]
Our S3 File Manager offers a lightweight, zero-configuration solution focused purely on S3 storage management, making it ideal for teams who need simple, cost-effective cloud storage without the overhead of a full collaboration suite like Nextcloud. While Nextcloud excels at comprehensive collaboration, our tool excels at simplicity and AWS integration. But we don't want to limit it to S3. We will try to include all storage by next month.
szszrk 20 days ago [-]
Hey, looks interesting and practical.
Any plans on making it compatible with other s3 implementation (other cloud vendors, local minio.io etc)?
rohitghumare 20 days ago [-]
Yes! Will release next version which supports all cloud providers from S3 to Hetzner Cloud storage!!
szszrk 20 days ago [-]
That is really neat. I had so many tiny use cases in my previous companies that could be solved by just a simple UI like this one. If you combine that with things like s3 static websites, it could be a beast that replaces some long-forgotten CMS solutions.
This is not that difficult if you're already invested in an identity ecosystem, but a right pain without something to bootstrap it.
On the plus side, AWS creds can be made to be temporary and limited in scope to just the nouns/verbs required. Creating and vending those tokens is an exercise for the reader.
And of course generating OIDC PKI JWTs is pretty easy and well documented elsewhere.
The harder parts in my mind are:
Cloning and installing is also an option.
There should be instructions on how to set that IAM user up (dont make it the root! It just needs full access to a single bucket ideally).
Give one of the implementations a try: https://github.com/psanford/wormhole-william
https://docs.aws.amazon.com/AmazonS3/latest/userguide/ShareO...
Google Cloud, Backblaze, Digital Ocean, Cloudflare, Azure all have this presigned URLs functionality too (I checked for the degree of lockin before I started using S3's presigned URLs in a set of bulk-data APIs at one place I've worked.)
GCP: https://cloud.google.com/storage/docs/access-control/signed-...
Digital Ocean: https://docs.digitalocean.com/products/spaces/how-to/set-fil...
Hetzner: https://docs.hetzner.com/storage/object-storage/faq/buckets-...
Backblaze: https://backblaze-prod.us.document360.io/apidocs/b2-get-down...
Cloudflare R2: https://developers.cloudflare.com/r2/api/s3/presigned-urls/
Azure: https://learn.microsoft.com/en-us/rest/api/storageservices/d...
etc.
Also, presigned URLs can be used not just for downloading files with a temporary URL but can also be created for uploading files with a temporary URL.
But it is not great if you want to distribute a file multiple times, asynchronously, or with other functionality gained from centralized storage. This is where people typically use email, dropbox, or perhaps the tool from TFA.
Non-free Resilio Sync basically automated this
Testing Other Cloud Providers:
Backblaze B2 ($5/TB/month)
Wasabi ($6/TB/month)
Google Cloud Storage ($20/TB/month) - I am ready to deploy to production.
DigitalOcean Spaces ($5/TB/month)
Cloudflare R2 ($15/TB/month)
Hetzner Storage ($3/TB/month)
There really isn’t a need for them. If you are running the application on any AWS compute - EC2, Lambda, ECS, EKS, etc, there is an IAM role attached to the VM that gives code permission to run.
On the client facing side, it should be connected to your Orgs SSO solution so when a person leaves the company, you deactivate the user in one place.
Besides, I can’t think of any organization of even 2 people that isn’t already using Office365/OneDrive or Google/GSuite with plenty of shared storage. The cost per seat for either is $6 - $25 per user.
As far as sharing files goes though, yes. I mean, it allows you to... share files, and do so in a controlled manner. Even edit them in-app as long as they're of a supported format.
Just published a new release to support multiple cloud storage options.
Thanks
> We support 7 cloud providers now. > Beta storage providers are still being tested.
Stable: Amazon S3 ($23/TB/month) Google Cloud Storage ($20/TB/month) Cloudflare R2 ($15/TB/month)
Beta/Testing: Wasabi ($6.99/TB/month) Backblaze B2 ($6/TB/month) DigitalOcean Spaces ($5/TB/month) Hetzner Storage ($4/TB/month)
Were you familiar with Filestash before starting this? If so, was there something specific you felt was missing that inspired you to create your own solution? Would love to hear your thoughts.
github: https://github.com/mickael-kerjean/filestash
demo on s3: https://demo.filestash.app/login?type=s3&access_key_id=Q3AM3...
Could you tell me why Google Drive didn’t work for you?
Google Drive seems to cheaper and has better UX than S3 - $8 for 2 TB (India) - Supports file versioning - 750GB bandwidth per day
What works bad: * folder with lots of medium size files * large ZIP * video bigger than a few tens of megabytes
Any plans on making it compatible with other s3 implementation (other cloud vendors, local minio.io etc)?