Spec the job and split it into smaller specialized cases and typed questions
Spec the job and split it: for Jev, that meant splitting some 800,000 rows into 200,000 smaller specialized cases, and each of those cases into typed questions.
40,153 results · Most relevant
Spec the job and split it: for Jev, that meant splitting some 800,000 rows into 200,000 smaller specialized cases, and each of those cases into typed questions.
Find an architecture that gives a solid result, use it as a baseline, then swap models and parts for faster, cheaper options, compare against known intermediates, and tighten the requirements for success each time.
Automate accounts receivable collections with an agent — it is a universal pain point that most companies still handle manually. The playbook: send the invoice to the correct finance point of contact within 10–30 minutes of the deal closing; check whether a finance contact is listed on the contract and email the line-of-business person as primary and CC the finance contact (or vice versa); if no finance contact is listed, use enrichment to look up and guess the right finance recipient (most companies have standard AP or finance inboxes); CC multiple stakeholders on every collections invoice (contract signer, deal POC, finance contact, line-of-business owner) so someone can always escalate; have the agent monitor the collections inbox daily; send a first payment-reminder touchpoint one week
Jake's site overhaul included a full rework of structure and navigation, and a review-page template: lead with a huge image, a very short intro, and a quick summary block capturing the searcher's intent; use your own brand colors so the visual pops; briefly cover pros and cons, pricing, rating score, and size and fit (fit being super important to shoe buyers); focus on attributes that matter — stability, versatility, durability, quality; include high-quality video of actually testing shoes and show your own experience wearing them to prove you're the real deal; and give readers the meat and potatoes above the fold — everything they need in their first five seconds — without assuming every reader wants to go as deep as the creator.
On a review page, the first thing readers should see is a huge image, followed by a very short intro and a quick summary of the entire post. The summary block represents the intent of the searcher: tell readers what people care about, go through the pros and cons very briefly, and include pricing, your rating score, and size and fit — size and fit being super important to anyone who actually wants to buy a shoe. The next thing readers do is qualify themselves, such as with high-quality video of the reviewer actually testing shoes, and show your own experience demonstrating that you're actually wearing the shoes you review.
Matt Ragone lays out a five-step method for building a skill: pick a workflow you repeat, teach the agent with real cases, pack it into a skill, test it on real cases, and ship it with an owner and a reviewer. None of the steps require writing code, but every one requires knowing the work really well.
The runtime authenticates with a security token service using its app client credentials or workload identity and sends the subject token containing the user's identity and access. It creates a token exchange request asking for permission to access the MCP server for that tool call — and only that tool call. Token exchange yields three pieces of info missing from the demo: the identity of the agent requesting access, the identity of the user it acts on behalf of, and the level of access the user granted the agent. Whether the token should be issued is decided using governance policy evaluated against the requested access, the target resource, the agent's identity, and the user's identity. When everything checks out, the STS issues an access token for the MCP server, keeping the user as the
Javier Garza outlines a practical AI threat modeling workflow: first discover and map the AI system structure (e.g., with an AI-generated map, converting tool JSON output into a graphical data flow diagram, or AI-aware architecture diagrams/model artifacts), then enumerate threats such as adversarial inputs and prompt injection to ensure complete coverage. He also stresses that threat models need to evolve alongside application deployments, and that threat modeling should be integrated early when designing AI-agentic workflows because it is proactive.
The core loop runs on a real agent: detect the issue (e.g., the price-filtering bug), investigate using the Arize trace tool or through Signal, fix the issue with Claude patching the fallback, then verify. Step one is detect — surface the fallback failure before a user reported it. Step two: based on that failure and evals running in your platform, investigate what's happening with real traces. In the example, Signal and the skills via Claude Code looked at traces of a user asking for the price under $30, married that to an investigation of the code path that checks the price filter, and proposed a fix. The speaker found a bug in his own price filter code by using Signal, the trace data in Arize, and the skills via Claude Code, seeing the interaction happen with a real user and then fixing
Run the full loop: use Arize skills to create a dataset of failed traces (e.g. ~25 budget-constrained search failures), run trace analysis with Claude Code to identify the bug and prepare a fix as a PR, stand up the dev agent behind an endpoint, hit it with the failure dataset, and score responses with the same judge rubric used in production. The agent can run this autonomously — collecting failures, creating a dataset, running it against a new version of itself, and preparing a report.
endorsed
With agent experiments and the agent playground, deploy the fixed agent to dev, stream dataset inputs into it, and compare side by side what the dev agent with the fix does versus what the production agent did. Look at before-fix vs after-fix results, the judge, examples and queries, and operational metrics like latency and cost — then quantify the results of the experiment.
endorsed
To get an agent to use a graph tool, don't fight the model's prior experience — lean into it by making the graph feel like what the model already uses. Rename graph query tools to 'search' (e.g., call a tool 'query graph' just 'search') so the agent uses it naturally; if you name a tool 'search', the agent will almost always do multi-keyword search into it with no training. Meet the model halfway: name the tool 'search', account for how the model will query it, and adjust the tool's behavior accordingly. Return graph results to the model in the same tool structure and response format as ripgrep, because that's what the model has been trained on.
The recommended agent-debugging loop: first identify production failures, then gather a nice sample size of that failure information for experimentation. Determine what actually was the issue — whether the LLM prompt should change, guardrails should be added, or the tool was doing the wrong things. Bring in evaluations to see where things went wrong, make the changes, run the experiment, ship to production, and repeat the whole loop continuously.
The standard process: (1) identify production failures such as wrong tool calls, (2) gather a nice sample size of failure information for experimentation, (3) determine what actually was the issue — prompt change, guardrails, or a broken tool, (4) build out evaluations to continuously monitor the use case, (5) make the changes, run the experiment, ship to production, and repeat the loop continuously. It works but is long, manual, and tedious.
LiveKit built the benchmark they wished existed: they take a complete working agent (the hotel receptionist that takes real bookings with a real backend) and it's open source. They run hundreds of simulated callers against the agent; the simulator interrupts, mumbles, and changes its mind just like a real caller would. They don't score on vibes; they score outcomes defined the way they would for a human: did the right reservation land in the database, did a caller get cut off, are the names and dates correct, and were there any invented tool calls or leaked personal data. Completion alone isn't enough as a benchmark metric because completion can be gamed; a model can guess its way to complete. They penalize what a completion score can't see, like asking questions that didn't need to beaskd
End-of-turn detection fires from audio before the transcript lands, so the language model is already running on a provisional transcript — if the final words match, you just saved a half a second. TTS should start on the first complete sentence, not the last token, because the first audible byte is when the human stops waiting. Every model you choose shows up in the gap between the caller's last syllable and the agent's first audible byte, and this window has a fixed size.
In-season training should work to maintain 85-95% of absolute strength — the first job across three sessions. Assign one back squat session per week, plus a heavy unilateral lower-leg exercise such as a front foot split squat or Bulgarian split squat. For athletes with weekend games, stack the back squat session at the start of the week (probably Monday morning) assuming the athlete is fairly fresh. Maintain upper body strength as well, though it likely has less negative influence on gameplay. The secondary goal is transference into speed and power: clean pulls from rack, power cleans, and faster, higher-speed, slightly lower-force exercises. Accessory exercises go in the latter half of the session and are predominantly prehab/rehab.
Patterson lays out a coherent security method: (1) control data access by running agents in a secure, preferably virtual environment rather than on your laptop — this blocks private file access and prevents destructive actions on your machine; (2) restrict external communication by putting the agent on a VM isolated from your network, only able to reach domains you want, on your VPN/VPC, through a firewall; (3) govern all agent traffic on the box so you can see everything it does; (4) set up alerting so admins are notified if an agent breaches the perimeter and can kill the agent; (5) lock down content, contact, and access with restricted permissions and tightly sculpted credentials.
endorsed
Patterson recommends using cloud development environments as a remote execution layer: access them via URL, RDP, or SSH rather than working locally; run agents on cloud infrastructure that is isolated with right-scoped permissions; configure the environment so agents cannot communicate with anything unless explicitly allowed, including the public internet; and pre-configure environments so agents can start developing immediately without downloading anything. In an isolated cloud environment, an agent can only work on the code, credentials, and data you purposely put into it.
endorsed
Patterson recommends using a model proxy between the agent/model on your machine and the LLM provider: route agent traffic through a proxy server or gateway that sniffs all traffic and logs everything the agent does; the proxy can sanitize traffic and strip prompt injections, and can strip sensitive data like accidentally included credit card numbers before prompts reach the LLM provider.
endorsed
Choose sources → to get a library of your own and a daily brief.