llama.cpp로 AI 에이전트 구축

apt-get update
apt-get install pciutils build-essential cmake curl libcurl4-openssl-dev git -y

# Clone llama.cpp repository
git clone 

# Configure build with CUDA support
cmake llama.cpp -B llama.cpp/build \
    -DBUILD_SHARED_LIBS=OFF \
    -DGGML_CUDA=ON \
    -DLLAMA_CURL=ON

# Build all necessary binaries including server
cmake --build llama.cpp/build --config Release -j --clean-first

# Copy all binaries to main directory
cp llama.cpp/build/bin/* llama.cpp/

./llama.cpp/llama-server \
    -hf unsloth/gemma-3-4b-it-GGUF:Q4_K_XL \
    --host 0.0.0.0 \
    --port 8000 \
    --n-gpu-layers 999 \
    --ctx-size 8192 \
    --threads $(nproc) \
    --temp 0.6 \
    --cache-type-k q4_0 \
    --jinja

(main) [email protected]:/workspace$ curl -X POST  \
    -H "Content-Type: application/json" \
    -d '{
        "messages": [
            {"role": "user", "content": "Hello! How are you today?"}
        ],
        "max_tokens": 150,
        "temperature": 0.7
    }'

{"choices":[{"finish_reason":"length","index":0,"message":{"role":"assistant","content":"\nOkay, user greeted me with a simple "Hello! How are you today?" \n\nHmm, this seems like a casual opening. The user might be testing the waters to see if I respond naturally, or maybe they genuinely want to know how an AI assistant conceptualizes \"being\" but in a friendly way. \n\nI notice they used an exclamation mark, which feels warm and possibly playful. Maybe they're in a good mood or just trying to make conversation feel less robotic. \n\nSince I don't have emotions, I should clarify that gently but still keep it warm. The response should acknowledge their greeting while explaining my nature as an AI. \n\nI wonder if they're asking because they're curious about AI consciousness, or just being polite"}}],"created":1749319250,"model":"gpt-3.5-turbo","system_fingerprint":"b5605-5787b5da","object":"chat.completion","usage":{"completion_tokens":150,"prompt_tokens":9,"total_tokens":159},"id":"chatcmpl-jNfif9mcYydO2c6nK0BYkrtpNXSnseV1","timings":{"prompt_n":9,"prompt_ms":65.502,"prompt_per_token_ms":7.278,"prompt_per_second":137.40038472107722,"predicted_n":150,"predicted_ms":1207.908,"predicted_per_token_ms":8.052719999999999,"predicted_per_second":124.1816429728092}}

export TAVILY_API_KEY="your_api_key_here"
export OPENAI_API_KEY=local

%%capture
!pip install -U \
    langgraph tavily-python langchain langchain-community langchain-experimental langchain-openai

from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    model="unsloth/gemma-3-4b-it-GGUF:Q4_K_XL",   
    temperature=0.6,
    base_url="         
)

from langchain_community.tools import TavilySearchResults
from langchain_experimental.tools.python.tool import PythonREPLTool

search_tool = TavilySearchResults(max_results=5, include_answer=True)
code_tool   = PythonREPLTool()

tools = [search_tool, code_tool]

from langgraph.prebuilt import create_react_agent

agent = create_react_agent(
    model=llm,
    tools=tools,
)

def extract_tool_names(conversation: dict) -> list[str]:
    tool_names = set()
    for msg in conversation.get('messages', []):
        calls = []
        if hasattr(msg, 'tool_calls'):
            calls = msg.tool_calls or []
        elif isinstance(msg, dict):
            calls = msg.get('tool_calls') or []
            if not calls and isinstance(msg.get('additional_kwargs'), dict):
                calls = msg['additional_kwargs'].get('tool_calls', [])
        else:
            ak = getattr(msg, 'additional_kwargs', None)
            if isinstance(ak, dict):
                calls = ak.get('tool_calls', [])
        for call in calls:
            if isinstance(call, dict):
                if 'name' in call:
                    tool_names.add(call['name'])
                elif 'function' in call and isinstance(call['function'], dict):
                    fn = call['function']
                    if 'name' in fn:
                        tool_names.add(fn['name'])
    return sorted(tool_names)

def run_agent(question: str):
    result = agent.invoke({"messages": [{"role": "user", "content": question}]})
    raw_answer = result["messages"][-1].content
    tools_used = extract_tool_names(result)
    return tools_used, raw_answer

tools, answer = run_agent("What are the top 5 breaking news stories?")
print("Tools used ➡️", tools)
print(answer)

Tools used ➡️ ['tavily_search_results_json']
Here are the top 5 breaking news stories based on the provided sources:

1.  **Gaza Humanitarian Crisis:** Ongoing conflict and challenges in Gaza, including the Eid al-Adha holiday, and the retrieval of a Thai hostage's body.
2.  **Russian Drone Attacks on Kharkiv:** Russia continues to target Ukrainian cities with drone and missile strikes.
3.  **Wagner Group Departure from Mali:** The Wagner Group is leaving Mali after heavy losses, but Russia's Africa Corps remains.
4.  **Trump-Musk Feud:** A dispute between former President Trump and Elon Musk could have implications for Tesla stock and the U.S. space program.
5.  **Education Department Staffing Cuts:** The Biden administration is seeking Supreme Court intervention to block planned staffing cuts at the Education Department.

tools, answer = run_agent(
    "Write a code for the Fibonacci series and execute it using Python REPL."
)
print("Tools used ➡️", tools)
print(answer)

Tools used ➡️ ['Python_REPL']
The Fibonacci series up to 10 terms is [0, 1, 1, 2, 3, 5, 8, 13, 21, 34].