PrivateGPT is a great starting point for using a local model and RAG.
Text-generation-ui, oogabooga, using superbooga V2 is very nice and more customizable.
I’ve used both for sensitive internal SOPs, and both work quite well. Private gpt excels at ingesting many separate documents, the other excels at customization. Both are totally offline, and can use mostly whatever models you want.
Not only is the demo funny, but this worked, surprisingly, as advertised. Had to restart the environment a few times for some reason. Not sure I understand the authors security concerns, but this is a fantastic early implementation.
I've found lowering the temperature and disabling the repetition penalty can help [0]. My explanation is that the repetition penalty penalizes the end of sentences and sort of forces the generation to go on instead of stopping.
So, I stumbled upon this Simple LLaMA FineTuner project by Aleksey Smolenchuk, claiming to be a beginner-friendly tool for fine-tuning the LLaMA-7B language model using the LoRA method via the PEFT library. It supposedly runs on a regular Colab Tesla T4 instance for smaller datasets and sample lengths.
The so-called "intuitive" UI lets users manage datasets, adjust parameters, and train/evaluate models. However, I can't help but question the actual value of such a tool. Is it just an attempt to dumb down the process for newcomers? Are there any plans to cater to more experienced users?
The guide provided is straightforward, but it feels like a solution in search of a problem. I'm skeptical about the impact this tool will have on NLP fine-tuning.
> I can't help but question the actual value of such a tool. Is it just an attempt to dumb down the process for newcomers?
Actually, you've hit the nail on the head here. I wanted something where I, a complete beginner, can quickly play around with data, parameters, finetune, iterate, without investing too much time.
That's also why I've annotated all the training parameters in the code and UI -- so beginners like me can understand what each slider does to their tuning and to their generation.
This is exactly the sweet spot I'm looking for. Technical enough that I can play around, simplified enough that I'm investing an hour or two of my time instead of a whole weekend.
maybe put the bit it said in quotes? I didn't read closely enough myself the first time, it took your subsequent comments to make me realize what you'd done
Chatgpt can generate pretty decent PLC structured text, IEC-61131 compliant code. Here, I’ve detailed how I generated an Analog Input processing Control Module. Rockwell IDE is used, but it will work for any similar processor.
I’ve used both for sensitive internal SOPs, and both work quite well. Private gpt excels at ingesting many separate documents, the other excels at customization. Both are totally offline, and can use mostly whatever models you want.