I installed OpenAI’s Symphony, by simply directing Codex to its GitHub repository, as recommended by Symphony’s README:
Launch codex in your repo, give it the URL to the Symphony repo, and ask it to set things up for you.
Before I run Symphony, I will need to first create the issues corresponding to the task I would like to complete. I used the following prompt:
Right now, there is no UI for a reviewer/TPC member to enter a review (or a meta review or a Area Chair review). The review form can already be configured by the chair under administration, but it did not surface to the reviewers or TPC members or Area Chairs. Each entry in the table in their list of “My Assignments” does not really have a button that they can use to view (and to review) the paper.
Use your frontend design skills, design such a UI for reviewers/Area Chairs/TPC members to enter reviews/meta reviews/Area Chair reviews, based on the configured templates from the ‘review form’ configuration under Administration.
Write a comprehensive plan carefully and break this into issues in project ‘reviewsdue’ in Linear. Scope each issue to one atomic task that can be completed with one agent and scoped to one reviewable PR. Include acceptance criteria in each issue. For each issue, add a follow-up ‘review’ issue that have acceptance criteria like:
- reviewer reads the full diff, not just the summary
- verifies behavior against the acceptance criteria of the issue
- checks tests are adequate and not just passing narrowly
- records concrete findings or explicitly states no findings
- only closes after review comments are addressed
This provides a deliberate “implementation issue” followed by a separate “review issue” workflow, which is better than trusting the implementation issue alone. Set blocking relationships where order matters; later issues should be blocked by both its upstream implementation issue and their review gate.
The first two paragraphs described the task I wish Codex to work on, and the later paragraphs instructed Codex to use Linear, and to create both implementation issues and their review gates.
Symphony runs correctly, but I found that it is using a lot of tokens for simple features, and does not allow me to see what is going on in each of the active sessions it launched. For one issue, it couldn’t finish running the agent after over an hour — something must be broken inside the agent.
So instead of depending on Symphony to pull the issues, I entered the following prompt:
Now that you have the Linear issues, pull each of these issues, respect dependency requirements, and then spawn subagents (in parallel, if needed) to resolve these issues. Once each agent is finished, commit with a detailed commit message following the PR instructions when pushing (but don’t need to PR, just commit). Respect instructions in WORKFLOW.md when pulling Linear issues. When all the issues have been completed, PR the entire feature with a detailed description.
This did the trick: it worked in a faster and perhaps more token-efficient way as using Symphony; but even more importantly, there is more transparency: I can read what the agents did in their sessions.
There is one major advantage of using this workflow and Linear: after each issue is completed, Codex would add results of running the agents as a comment in the issue itself, providing me with a central repository to log the history of all the work completed by the agents in the project. I think this is more important than the speed of running multiple agents, since I can read these comments and get a sense of what’s going on, reducing the cognitive debt when managing the project.