Building KRXY — Part 4: When a Model Name Stops Being Enough

Part 3 ended with a Raspberry Pi that could identify itself, report its health, and make a signed claim about what it could do. The next question was whether that claim should decide where a job went. On 25 October 2026, KRXY began making that distinction. A node being online was no longer enough. It had to be able to run the particular model a consumer had requested.

This part covers that change and the work that followed during the morning and afternoon of 25 October. There was still only one physical Raspberry Pi. The second node, the larger evaluation runs, and the distributed-training results belong to later entries. The evidence here comes from the development log, the node runbook, the capability-scheduling decision, and the recorded live checks.

From an online node to a compatible deployment

Until this change, the gateway knew about a Hailo-hosted Qwen2.5 1.5B deployment. With only one real provider online, it could send a request there without making much of a placement decision. That was useful for commissioning, but it would not scale to a network where different nodes offered different runtimes and model files. Merely knowing that a Raspberry Pi was online did not tell the gateway which model it could execute.

The new placement rule matched the requested deployment against capabilities reported by a live node. The deployment catalogue recorded what the platform expected, while the agent reported what its running environment supported. A consumer's model choice was now a constraint, not a suggestion. The CPU diagnostic model was being prepared alongside the existing Hailo model, but preparation was not the same as availability.

On the first rebuilt gateway check that morning, the running agent was still 0.3.2-hailo. The model endpoint listed only hailo:qwen2.5-instruct:1.5b. A short request completed and returned Ready.. The newer agent files, CPU environment configuration, and a local llama-server process were already present on the Pi, but restarting the agent needed a local sudo password. The running process remained unchanged, so it did not advertise the CPU deployment. This was a repeat of the lesson from Part 3: the code on disk is not necessarily the code answering requests.

Why a model name was not enough

A name such as Qwen2.5 1.5B identifies a model family and size, but not a particular executable artifact. The Hailo path used a compiled HEF through hailo-ollama. The CPU path used a pinned Q4_K_M GGUF through llama.cpp. They were different artifacts, running through different software, even though both were based on published Qwen2.5 models.

The next routing change required a hosted-model claim to include its model ID, the catalogue runtime, and a 64-character artifact digest. That gave the gateway a stricter basis for matching a job to a provider. An existing enrolled Hailo node received a temporary legacy exception during the transition; it was not a general permission for nodes to claim models without evidence of the expected artifact. The agent's signed identity established which node made a claim. It did not independently prove that the claim was honest or that the model output was correct.

The difference became visible when the gateway received a request for cpu:qwen2.5-1.5b-instruct:q4_k_m while the live agent still reported only Hailo. The gateway returned no_compatible_node. It did not replace the CPU model with Hailo or fall back to the development simulator. Unsupported sampling options were rejected rather than quietly ignored. That refusal was a useful result: a plausible answer from the wrong model would have been a routing failure, even if the text looked good.

Making the CPU deployment available

Later that afternoon, the local sudo prompt on the Pi was accepted. The CPU service, krxy-cpu.service, started, and the running agent became 0.3.4-hailo. The node then advertised both hailo:qwen2.5-instruct:1.5b and cpu:qwen2.5-1.5b-instruct:q4_k_m. The gateway could see two hosted deployments on one provider, without treating them as two physical nodes.

The first live checks were deliberately modest. A Hailo request returned Ready. on the Hailo deployment ID, and a CPU request returned Ready. on the CPU deployment ID. The consumer page kept Hailo selected by default while making the CPU option available. These short responses checked routing and execution, not whether one model wrote better answers than the other. The CPU llama-server listened on the Pi's loopback interface. No second Raspberry Pi or remote CPU worker had been added.

The hosted jobs still had verification marked unsupported and reserved credits of 0. They produced no provider contribution or usage-ledger entry. Both the HEF and the GGUF were published model artifacts, not models trained by KRXY. A completed inference request was evidence that the route worked, not proof of model integrity, useful contribution, or a claim to payment.

Removing a capability without losing the node

The next test asked whether the catalogue would keep showing a model after its runtime had stopped. Only krxy-cpu.service was stopped. On the next catalogue read, the CPU deployment disappeared, and a request for it returned no_compatible_node. Hailo remained available and answered a separate request. Starting the CPU service again restored its model entry, and another CPU job completed.

This mattered more than simply proving that both runtimes could start. The model list now reflected a live capability instead of a record that software had once been installed. If the CPU runtime went away, its deployment went away with it; the Hailo route did not have to disappear as well. For a future network with mixed hardware and intermittent availability, the scheduler needs to distinguish a node that is connected from a specific deployment that is ready to take work.

Two runtimes, one busy Raspberry Pi

With both deployments present, the next check sent overlapping Hailo and CPU requests. Hailo succeeded, while the CPU request returned node_busy. The agent still allowed one in-flight generation on the provider node. Having two local inference processes did not automatically make them two independent workers, and the scheduler did not count them that way.

This was a conservative limit, but an intentional one. Running several models on the same machine is not equivalent to adding another machine to the network. The Pi's system memory, CPU and other resources are shared. Until concurrent capacity is measured and represented properly, a clear busy response is preferable to accepting work that the agent cannot safely run.

A final check repeated a request using the same idempotency key. The gateway returned the same job rather than creating a second one. Retries are normal in distributed systems: a client may lose its connection after a request has been accepted, or may send the request again because no reply arrived. Treating a retry as new work could eventually produce duplicate executions and duplicate accounting. That did not happen in this check.

What the day established

By the end of 25 October's routing work, KRXY could select an exact hosted deployment, refuse an incompatible request, show two runtimes on one node, withdraw the CPU capability when its service stopped, report a busy provider, and recognise a repeated job request. The important change was not the addition of a CPU model by itself. It was that a reported capability had become something the gateway used to make and enforce a placement decision.

There were limits to the result. Only one physical Raspberry Pi was operating as a provider. The model claims were not independently attested, the hosted responses were not verified contributions, and no hosted contribution credits were paid. Open WebUI was not signed in for the final activation checks. A training baseline also ran on the Mac that day, but its predeclared comparison gate was not met; that research has a separate story and is not evidence of successful decentralised training.

The next question followed naturally. Routing a request to the right model did not tell me whether that model was any good at the task. The evaluation work later on 25 October began measuring answer quality separately from the scheduler's ability to deliver a request to the intended runtime. That is where Part 5 will begin.