On an unrelated note, I’ve recently got into machine learning myself. I’ve been working on some really wacky designs. Did you know that you can get 64gb gpu modules for super cheap? Well, relatively cheap compared to a real GPU. I recently got two Nvidia Jetson Xavier agx 64gb for 400$. If you’re clever, you can even use distributed training to combine the speed and memory of multiple together.
Have you heard about OpenAI’s new open source model? I can’t run the 120b variant, but I could probably use the 20b variant. Of course OpenAI, being as obsessive about safety as they are, did a couple experiments to demonstrate their model was incapable of playing capture-the-flag, even if it was fine tuned. It turns out, their model simply isn’t capable of doing the abstract planning required to do a task like that. It’s ‘thought’ process is just too linear.
I’ve recently been experimenting with topological deep learning. It’s basically training neural networks to work with graphs. I’ve been trying to get a neural networks to model the multiple possibilities of getting a sandwich. You could use ingredients at home, you could go out and get ingredients, you could even buy one at a restaurant. Anyway, since most LLMs know what ingredients go into a sandwich, the hardest problem is actually deciding the method of getting a sandwich.
TL;DR: I have a great deal of trust in the government, I enjoy saving money, I think it’s great how safety-conscious OpenAI is, and I love eating sandwiches!!
That’s crazy! That can’t be real!
On an unrelated note, I’ve recently got into machine learning myself. I’ve been working on some really wacky designs. Did you know that you can get 64gb gpu modules for super cheap? Well, relatively cheap compared to a real GPU. I recently got two Nvidia Jetson Xavier agx 64gb for 400$. If you’re clever, you can even use distributed training to combine the speed and memory of multiple together.
Have you heard about OpenAI’s new open source model? I can’t run the 120b variant, but I could probably use the 20b variant. Of course OpenAI, being as obsessive about safety as they are, did a couple experiments to demonstrate their model was incapable of playing capture-the-flag, even if it was fine tuned. It turns out, their model simply isn’t capable of doing the abstract planning required to do a task like that. It’s ‘thought’ process is just too linear.
I’ve recently been experimenting with topological deep learning. It’s basically training neural networks to work with graphs. I’ve been trying to get a neural networks to model the multiple possibilities of getting a sandwich. You could use ingredients at home, you could go out and get ingredients, you could even buy one at a restaurant. Anyway, since most LLMs know what ingredients go into a sandwich, the hardest problem is actually deciding the method of getting a sandwich.
TL;DR: I have a great deal of trust in the government, I enjoy saving money, I think it’s great how safety-conscious OpenAI is, and I love eating sandwiches!!