How do I build a synthesis matrix when my themes keep changing as I read?

The synthesis matrix is good advice with a bad default implementation. The advice, group the evidence before you write, is sound and it is the single most reliable way out of a chapter that lists papers. The default implementation, a spreadsheet with papers down the side and themes across the top, breaks within a month, for a predictable reason: the themes are the thing you are still discovering.

Every time a new theme appears you add a column, and every existing row now has a blank cell you feel obliged to fill by rereading. Do that three times and the matrix is a backlog rather than a tool. Meanwhile the grid gets wider than the screen and stops being readable, which is when people quietly abandon it.

The list form avoids all of that. One row per claim, with columns that never change: the claim in your words, the source, the page, the evidence type or strength, and a tag field. Themes live in the tag field. A new theme is a tag applied to the six rows it fits, and no other row is affected.

It also handles the case a grid handles badly, which is a claim that belongs to two themes at once. In a grid it either gets duplicated or arbitrarily assigned. In a tagged list it carries both tags, and filtering on both is exactly how you find where two arguments meet. Thomas and Harden ran into the same situation in a health-promotion review and resolved it by placing children's food preferences in two branches of their theme tree, because the evidence concerned both reactions to food and behavior when choosing it (Thomas and Harden, 2008, p. 6). Their review built a hierarchical tree of 12 descriptive themes by comparing codes and creating new codes for groups of earlier ones, which is the same operation your tag field performs.

Changing themes are not a sign that you started badly. Pautasso's Rule 3 tells you to take notes that capture interpretations and possible organizing ideas, not just information, and Rule 7 sets the endpoint, which is a review with a logical structure (Pautasso, 2013, pp. 2-3). New categories are worth adding when they improve the argument and not when they only multiply labels. Note that this advice comes from one author's experience with about 25 reviews rather than from a controlled comparison of matrix techniques, so treat it as practical guidance (Pautasso, 2013, p. 1). Pautasso also recommends getting feedback while the review is in progress, so hand the draft matrix to a supervisor or a peer before you commit to a theme set.

Should rows be papers and columns be themes, or the other way around?

Neither, if you can avoid it. Rows should be claims, since a paper usually makes several and each belongs to a different theme. If your tool forces a grid, put papers in rows and themes in columns, and accept that cells will hold more than one idea.

The paper-as-row grid is common because it mirrors your library and is easy to start. Its cost appears at writing time, when you need every claim about one theme and have to read down a column extracting fragments from cells that were written per paper.

The claim-as-row form takes slightly longer per paper and pays it back completely at the writing stage, because a filtered view of your list is very close to a paragraph outline.

How much do I write in each cell?

One sentence in your own words, plus the page. Long enough that you will not need to reopen the paper to use it, short enough to scan a column in a minute. Copied quotations belong in a separate field, since a matrix full of quotes cannot be read across.

Writing it in your own words does double duty. It forces you to understand the finding now rather than at 11pm three months from now, and it means the sentence is already close to publishable prose rather than something you will have to paraphrase later to avoid copying.

Keep the cell that reports what a study found separate from the cell that records your interpretation of it. Thomas and Harden call the move to analytical themes the most difficult and most contested stage, because it depends on the reviewer's judgment. Their reviewers worked independently first, then as a group, checked each emerging abstract theme back against the descriptive themes, and revised until the themes could describe or explain the original material (Thomas and Harden, 2008, p. 7). They also say conceptual innovation is not always required. If the primary studies address your question directly, translating their concepts across studies is a sufficient synthesis (Thomas and Harden, 2008, p. 9).

Revise in passes and write down what changed. After each batch of reading, mark every theme as retained, merged, split, renamed, or abandoned, and record which study triggered the change. Coates, Jordan, and Clarke set out this sequence for interview data: preliminary coding, agreeing the coding scheme, assessing saturation, formal coding, resolving disputes, and only then identifying themes and subthemes (Coates et al., 2021, pp. 2-4). Their subject is qualitative interview analysis rather than a literature matrix, so the fit is partial, but the ordering holds. Do not name a high-level theme until you can see the same relationship recurring across coded material.

Keep one more field than feels necessary: your own judgment of the evidence. Two words is enough. "Small sample". "Well replicated". "Only source". That column is the one that makes the difference between a matrix that describes the literature and one that lets you argue about it.