Currently, the College of Letters & Science IT (CIT) offers access to LLM models via an API as well as the supported chat interface. Generally API access is really useful for more complex workflows and leveraging your own agentic tools.
API access is ideal for the following situations:
- Setting up your own tools.
- Running workflows off your own devices.
- Empowers agentic workflows.
- Training your own RAG, LoRA, or specialized models.
It does not give you access to:
- Add or run your own models
- No direct access to GPU/Cuda resources.
- No historical information or data is saved. Use is generally ephemeral unless you've configured your tools to record or track it.
While CIT doesn't support those other external tools, we do support the most complex and costly portion of it, the API level access to LLM models. There are lots of reasons to leverage CIT's API access to models, some of them include:
- OpenSource Models are used. This means data and results are more reproducible and available to other parties, while not being dependent on vendor lockin or descisions. OpenSource ensures that you retain rights to that model in perpitutity, ensuring your workflow remains accessible.
- Data and ideas stay at the institution. Data sovereingty is a continuing converstaion that researchers are having and exploring when it comes to AI. Even with agreements in place to not use any of the data to train future models, that's not the same thing as an agreement to not use the data at all. Itellectual Property, once shared, becomes very difficult to retract or retain and one must absolutely trust that the third party is working in your best interest.
- Cost - When you're starting off with AI, it's not uncommon to make mistakes that can severily impact output and cost. A bad prompt, poor instructions, or a bad loop can easily spend millions of output tokens in a few minutes in a permanent loop. With LLM Vendors, that could result in a quite a large bill. While we track token useage to ensure that all our users have equitible access to the service, we don't use hard limits, spend/token allowances, or charge for token useage.
- Flexibility - With API access you have full control over what prompts, settings, data, tools, and values are used specifically. We ship the model and you are enabled to control everything else.
You'll need an API key!