APAS alternatives - EleutherAI

[Michael McMahon]

Hi, Bradley & ai-assist!

I have a minor correction from the "Re: Minutes, APAS as a guerilla 
tactic for liberating proprietary software" thread.

 > APAS's are completely nascent right now; CoPilot is the only
 > one (so far) and by most accounts — as a product*today*  — it's more of a
 > curiosity than a useful tool.

CoPilot is the most well-known APAS, but it is not the only one.

EleutherAI is a organization that seems to be picking up where OpenAI 
left off before Microsoft stepped in.  While not branded as APAS, 
EleutherAI [1] released text generators that can act as AI code 
generators that are licensed under free licenses such as GPT-Neo [2], 
GPT-J-6B [3], and GPT-NeoX-20B [4] that are also trained on GitHub.

I would say the only functional differences between Copilot and 
EleutherAI models are that EleutherAI clearly defines their training set 
(called the Pile [5]), publishes their models under permissive licenses, 
their models and tools can be run locally offline, and EleutherAI has 
not published a extension to link their model with a text editor.

If you have the hardware, you can try the EleutherAI models out for 
yourself for a hands-on approach without using CoPilot.  I would suggest 
starting out with their smallest model GPT-Neo-125M with Hugging Face 
Transformers [6] as a sort of "Hello, World!" if you are new to this 
area of computing.  The smaller models can be run without nonfree 
software if you have approximately twice the amount of RAM as the size 
of the model.  Work your way up through the models to identify where 
your hardware limit is.

Both CoPilot and EleutherAI have the same issues with copyright inputs 
becoming public domain outputs, but these licensing issues with 
EleutherAI do not stem from public statements from EleutherAI as far as 
I understand it.  I am not a lawyer.

[1] https://www.eleuther.ai/
[2] https://www.eleuther.ai/projects/gpt-neo/
[3] https://www.eleuther.ai/projects/mesh-transformer-jax/
[4] https://www.eleuther.ai/projects/gpt-neox/
[5] https://pile.eleuther.ai/
[6] https://huggingface.co/EleutherAI/gpt-neo-125M

Best,
Michael McMahon | Web Developer, Free Software Foundation
GPG Key: 4337 2794 C8AD D5CA 8FCF  FA6C D037 59DA B600 E3C0
https://fsf.org

US government employee? Use CFC charity code 63210 to support us through the
Combined Federal Campaign. https://cfcgiving.opm.gov/