i find it curios llama-server treats a model as cache.
if you are going down the local model rabbithole you are going to want to save and organize these as production artifacts. they are nondeterminany entities so if one produces that sweet spot in use case, in production, you dont change it.
i get that we treat cloud infrastructure as always on but there needs to be a bit more conceptual forethought in value of determinant (eg, this model) when its use case produces the results you want.
We still us dbs and OSes decades old because theyre known quantities.
sure, imagine techno jesus will change that, but the model wont change.
if you are going down the local model rabbithole you are going to want to save and organize these as production artifacts. they are nondeterminany entities so if one produces that sweet spot in use case, in production, you dont change it.
i get that we treat cloud infrastructure as always on but there needs to be a bit more conceptual forethought in value of determinant (eg, this model) when its use case produces the results you want.
We still us dbs and OSes decades old because theyre known quantities.
sure, imagine techno jesus will change that, but the model wont change.