Rendered at 17:33:43 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
plantain 1 hours ago [-]
You are completely mad if you are running any kind of valuable workload on Runpod community or Vast. Hosts are absolutely peeking inside the VM's (you can see it openly discussed in their Discords).
gpm 1 hours ago [-]
There's got to be a ton of valuable workloads that don't care much about secrecy.
Who cares if someones spying on the process of fine tuning a model to help us pick strawberries? Even if they could somehow find a competitor to sell it to - the competitor wouldn't be interested because the risk of violating trade secret law is surely greater than whatever they can glean from our training run...
crazygringo 27 minutes ago [-]
Exactly. I've recently been training a number of models, and the value to any potential competitor would be approximately zero.
It's so integrated with the specific inputs of my product, and the outputs that it needs, that the model just has no value to anyone else.
If a competitor needed such a model, they need to train their own that matches the form of their inputs and outputs, which is just going to be different.
tgsovlerkhgsel 13 minutes ago [-]
As long as your training data itself isn't sensitive. If that's e.g. photos of products and you're training a quality assurance photo recognition software, that's probably fine, if it's unredacted support tickets, not fine.
Aurornis 29 minutes ago [-]
> Who cares if someones spying on the process of fine tuning a model to help us pick strawberries?
Privacy should be a default. It doesn’t matter what you’re doing.
It’s also unlikely they know what’s in the VMs until after they look at it.
tgsovlerkhgsel 14 minutes ago [-]
Sure, but if you have the option of training your strawberry picking model for 1/5th of the price knowing that you will be spied upon, that's probably still a good deal.
Just like you can find all of my 3D models online because not being able to make them private is the price for the free license of Onshape, and I'm happy to let you look at those models rather than pay whatever a commercial license would cost.
behnamoh 6 minutes ago [-]
> a competitor to sell it to - the competitor wouldn't be interested because the risk of violating trade secret law is surely greater than whatever they can glean from our training run
Unless the competitor is in a country like China.
elevation 40 minutes ago [-]
> violating trade secret law
There is law for patents. Trade secrets exist outside the realm of enforcement.
polishTar 14 minutes ago [-]
That's a common misunderstanding. If you learn of a trade secret through improper means, you can be liable federally under the DTSA or similar state statutes like CUTSA in California.
These were the statutes cited in the Anthony Levandowski lawsuit 9yrs back, where Uber had to give up a 0.34% equity stake in their company through the settlement.
> (you can see it openly discussed in their Discords).
Can you explain? I haven’t used Runpod and don’t have time to search through a Discord to figure out what someone is talking about.
dv_dt 50 minutes ago [-]
I hold the inverse risk assessment- I think acting on a peek risk is much higher with larger operations such as OpenAi or eg Amazon -as they've allegedly done the same on their product marketplace and deciding on generic products, in addition to open source-alike service offerings.
Larger entities have the resources, the inclination and the protective legal teams to act much differently than a resource tight n-tier platform operator
crazygringo 30 minutes ago [-]
It's really quite the opposite.
Large firms have a much harder time hiding that kind of behavior, and they have much more to lose.
Large firms require processes and controls around everything. If Amazon is peeking into corporate customers' data, then there have to be mechanisms designed to allow that, a whole bunch of people know, it's part of engineering designs, and a whistleblower is going to leak it to the press.
And then Amazon is going to lose all of its business customers once its privacy guarantees were shown to be intentionally circumvented.
It is simply not economically rational for a company like Amazon or OpenAI to look at the data of its corporate customers.
Regular consumers are a different story, because you need to respond to abuse, illegal activity, etc. So they don't have the same privacy guarantees.
But corporate customers, that stuff is watertight. No cloud provider is going to risk losing their entire business because they violated their privacy guarantees.
swiftcoder 15 minutes ago [-]
> If Amazon is peeking into corporate customers' data, then there have to be mechanisms designed to allow that, a whole bunch of people know, it's part of engineering designs, and a whistleblower is going to leak it to the press
An awful lot of this sort of thing in the corporate world is based on pinky-promises and legal threats, more than on robust technical controls.
Every production system I have ever seen has at least one set of dev-accessible keys that has access to the production DB. Mature systems at least had audit logging around who retrieved those keys, less mature systems often didn't have any kind of internal access logs.
Is "Amazon the corporate entity" snooping on your data? Absolutely not. Is someone with Amazon internal access snooping on your data? You'll never know either way.
winddude 6 minutes ago [-]
That's exactly how amazon decides what becomes an amazon basics product.
AWS has also strip mined open-source software, elasticsearch, redis, mongo, browser rendering tools, etc.
bojangleslover 6 minutes ago [-]
Could confidential computing help here?
kirubakaran 1 hours ago [-]
Could you please share screenshots?
m00x 59 minutes ago [-]
A lot of Vast.ai GPUs are also firmware-limited, so it's a big gamble when you run your workloads. You also can't get a cluster.
It's fine for comfyui workloads or simple LLM inference, but you lose both privacy and stability.
aunty_helen 51 minutes ago [-]
GPUs set to lower power states and CPUs with frequency scaling turned down. That's the biggest hit and miss for what I'm doing. Honestly don't care if people are looking at my workloads as it's just processing public data. But do care if it's someone's laptop that's actively using 1/2 the vram.
smallmancontrov 1 hours ago [-]
Sam is welcome to peek at my generative furry porn all he wants ( ͡° ͜ʖ ͡°)
tarpitt 58 minutes ago [-]
( ͠° ͟ʖ ͠°)
mmnfrdmcx 46 minutes ago [-]
What are places in your view to rent GPUs that are trustworthy they will not peek in your opinion?
WildGreenLeave 8 minutes ago [-]
UpCloud's [0] spot instances sometimes match the pricing of Vast.ai instances. I mostly use Vast.ai if my dual 3090s can't handle it anymore for non private things and move to UpCloud spot instances if it needs more.
There's a lot of use cases that are not "commercially sensitive" that would require data center level privacy. E.g. a good percentage of academic research and experimentation, long running "boilerplate processing" for monitoring logs for e.g. a bird watching system or large scale web crawling where the individual website itself is mostly public. There are also use cases where the data itself is useless without other data to correlate. E.g. many log analytics use cases, if you are missing the main database data itself then the logs have zero value when intercepted. There are also use cases where you are doing general industrial level hotdog/not hotdog classification where unit economics is the most important and there's no real value to privacy e.g. on a factory floor (though sabotage is a bigger risk here).
stale2002 33 minutes ago [-]
You are ignoring the real usecases here. Not everything is large scale, secretly, b2b enterprise applications. Instead, many consumer applications are fully fine with being out in the open.
As an example, midjourney use to just publish ever generated image to their website.
If you are just doing inference things are public/open from the start, you dont have to worry much about the stuff you are talking about here.
swiftcoder 13 minutes ago [-]
> As an example, midjourney use to just publish ever generated image to their website.
Which is kind of genius, if you think about it. Users can't get away with generating smut, CSAM, or copyrighted material, because it goes straight to the public record.
matt3210 5 minutes ago [-]
These prices don't seem real. For me, runpod is 8.99 an hour for b300 right now.
buredoranna 47 minutes ago [-]
There are a couple of new indices out there.
[1] "Silicon Index". Built against pricing observations, including listed prices and transaction data.
[2] "Ornn Compute Price Index (OCPI)". Claiming to use only executed transactions.
Your mileage may vary, but I find these new indices to be quite compelling.
Who cares if someones spying on the process of fine tuning a model to help us pick strawberries? Even if they could somehow find a competitor to sell it to - the competitor wouldn't be interested because the risk of violating trade secret law is surely greater than whatever they can glean from our training run...
It's so integrated with the specific inputs of my product, and the outputs that it needs, that the model just has no value to anyone else.
If a competitor needed such a model, they need to train their own that matches the form of their inputs and outputs, which is just going to be different.
Privacy should be a default. It doesn’t matter what you’re doing.
It’s also unlikely they know what’s in the VMs until after they look at it.
Just like you can find all of my 3D models online because not being able to make them private is the price for the free license of Onshape, and I'm happy to let you look at those models rather than pay whatever a commercial license would cost.
Unless the competitor is in a country like China.
There is law for patents. Trade secrets exist outside the realm of enforcement.
These were the statutes cited in the Anthony Levandowski lawsuit 9yrs back, where Uber had to give up a 0.34% equity stake in their company through the settlement.
https://en.wikipedia.org/wiki/Trade_secret#United_States
Can you explain? I haven’t used Runpod and don’t have time to search through a Discord to figure out what someone is talking about.
Larger entities have the resources, the inclination and the protective legal teams to act much differently than a resource tight n-tier platform operator
Large firms have a much harder time hiding that kind of behavior, and they have much more to lose.
Large firms require processes and controls around everything. If Amazon is peeking into corporate customers' data, then there have to be mechanisms designed to allow that, a whole bunch of people know, it's part of engineering designs, and a whistleblower is going to leak it to the press.
And then Amazon is going to lose all of its business customers once its privacy guarantees were shown to be intentionally circumvented.
It is simply not economically rational for a company like Amazon or OpenAI to look at the data of its corporate customers.
Regular consumers are a different story, because you need to respond to abuse, illegal activity, etc. So they don't have the same privacy guarantees.
But corporate customers, that stuff is watertight. No cloud provider is going to risk losing their entire business because they violated their privacy guarantees.
An awful lot of this sort of thing in the corporate world is based on pinky-promises and legal threats, more than on robust technical controls.
Every production system I have ever seen has at least one set of dev-accessible keys that has access to the production DB. Mature systems at least had audit logging around who retrieved those keys, less mature systems often didn't have any kind of internal access logs.
Is "Amazon the corporate entity" snooping on your data? Absolutely not. Is someone with Amazon internal access snooping on your data? You'll never know either way.
AWS has also strip mined open-source software, elasticsearch, redis, mongo, browser rendering tools, etc.
It's fine for comfyui workloads or simple LLM inference, but you lose both privacy and stability.
[0]: https://upcloud.com/
As an example, midjourney use to just publish ever generated image to their website.
If you are just doing inference things are public/open from the start, you dont have to worry much about the stuff you are talking about here.
Which is kind of genius, if you think about it. Users can't get away with generating smut, CSAM, or copyrighted material, because it goes straight to the public record.
[1] "Silicon Index". Built against pricing observations, including listed prices and transaction data.
[2] "Ornn Compute Price Index (OCPI)". Claiming to use only executed transactions.
Your mileage may vary, but I find these new indices to be quite compelling.
[1] https://www.silicondata.com/products/silicon-index
[2] https://data.ornn.com/
And yup, its not just how much these resources cost, but how much people are willing to pay for them.
(edit: formatting)
> What the 4.57× is not
> - InfiniBand
> - A datacentre SLA
> - Enterprise compliance, committed capacity...
ok
I’ve run into many hosted GPUs with processes stuck, with outdated drivers/libraries, or poor Internet connections
last time I was trying to run some workloads there the pricing of an h100 was largely irrelevant.