: Use the CLI to download a model like Llama 3 or Mistral: ollama pull llama3 .
Any Java framework interacting with Ollama relies on standard HTTP clients to exchange JSON payloads with this local server. This setup offers three key operational advantages: : Eliminates pay-per-token API fees.
: Building services that use "tool calling" to perform tasks like checking the weather or searching a database. ollamac java work
: If deploying to a test or staging environment, run Ollama inside a Docker container configured to utilize host GPU drivers for consistent scaling.
import dev.langchain4j.model.ollama.OllamaChatModel; public class LocalAiApplication public static void main(String[] args) // Initialize the Ollama model client OllamaChatModel model = OllamaChatModel.builder() .baseUrl("http://localhost:11434") .modelName("llama3") .temperature(0.7) .build(); // Generate a response String prompt = "Explain the concept of Dependency Injection in Java in two sentences."; String response = model.generate(prompt); System.out.println("AI Response:\n" + response); Use code with caution. 3. Asynchronous Streaming Responses : Use the CLI to download a model
For more advanced scenarios (RAG, agents, chains), LangChain4j is the go-to library for Java. It has built-in support for Ollama.
import org.ollamac.model.OllamacModel;
import dev.langchain4j.model.ollama.OllamaChatModel; public class LocalAiAssistant public static void main(String[] args) // Configure the model to match what you downloaded in Ollamac OllamaChatModel model = OllamaChatModel.builder() .baseUrl("http://localhost:11434") .modelName("llama3") .temperature(0.7) .build(); System.out.println("Thinking..."); String response = model.generate("Explain the benefit of running local LLMs in one sentence."); System.out.println("\nAI Response:"); System.out.println(response); Use code with caution. Method 2: The Lightweight Approach (Spring AI)
@Tool("Get the current weather for a given city") public String getWeather(@P("City name") String city) // ... call to a weather API return "It's currently 72°F and sunny in " + city; : Building services that use "tool calling" to
curl http://localhost:11434/api/generate -d ' "model": "llama3", "prompt": "Provide the name and population of France in JSON.", "format": "json", "stream": false '