Blog |

AI Research Assistant: Use Cases (5/5)

Fri, 31 Jul 2026 00:00:00 +0000

In this post, now that we have our AI assistant setup complete, I will showcase three use cases that are enabled by this specific setup we’ve been working towards. In each case I’ll outline the use case and then give an example based on my work, followed by illustrating how the information and knowledge flowed from our sources in Obsidian and Zotero to the AI assistant and from there into the answer given by the AI assistant. This is not about getting any answers from the AI, it’s about getting answers grounded in your work. After these use cases I will cover some temptations to resist when using such a setup and outline some directions I am considering for future development. All examples in the following were generated using GLM 5.2.

Use Case 1: Claim Verification

The first use case I would like to present is one of academic integrity. In our writing we rely on other sources and every time we cite a source we should be accurately representing the source we’re citing. While that sounds admirable in principle, in practice that is hard to do, given the wide range of sources we often cite. However, such a task is easy for an AI to perform. Let me illustrate this with an example. The fragment I’m using here for this example is from a of mine. This example is purposefully short as it’s only chosen to illustrate this type of usage.

Prompt to the AI:

I am working on a paragraph for a paper. Check that I am citing each source correctly. This is the text:

A challenge in video-based learning is mind wandering, broadly defined as thoughts unrelated to the current task. Mind wandering is a frequent cognitive phenomenon, with a recent meta-analysis showing that students engage in mind wandering approximately 30% of the time during learning activities (Wong et al., 2022). These frequent episodes of mind wandering have been shown to negatively impact on learning outcomes (Wong et al., 2022).

Following this prompt the AI assistant made some tool calls to access my Zotero database and then provided me with the following answer:

Show AI response (Claim Verification)

I found the source. Here is my verification:

Source identification

The cited source matches the paper in your Zotero library:

Title: Task-unrelated thought during educational activities: A meta-analysis of its occurrence and relationship with learning
Authors: Wong, A. Y., Smith, S. L., McGrath, C. A., Flynn, L. E., & Mills, C.
Year: 2022
Journal: Contemporary Educational Psychology, 71, 102098
Citekey: @wongTaskunrelatedThoughtEducational2022

Citation format check

“Wong et al., 2022” is correct. With five authors, APA 7 uses “Wong et al.” from the first citation onward. The year is correct.

Content / claim verification

Claim 1: “students engage in mind wandering approximately 30% of the time during learning activities”

This is accurate. The abstract states: “students are off-task about 30% of the time during educationally relevant activities.”

One small terminological note: the paper uses “task-unrelated thought” as its primary term (with mind wandering as a parenthetical gloss). If precision matters for your framing, you could note that the 30% figure specifically pertains to task-unrelated thought in educational contexts. But “mind wandering” is defensible given the paper explicitly equates the two in its title framing.

Claim 2: “These frequent episodes of mind wandering have been shown to negatively impact on learning outcomes”

This requires a correction. The meta-analysis found a negative correlation (r = −0.27, a small-to-medium effect), not evidence of causal impact. “Negatively impact on” implies causation that a meta-analysis of predominantly correlational studies cannot establish. Suggested rewording:

“These frequent episodes of mind wandering are associated with poorer learning outcomes (Wong et al., 2022).”

or, more precisely:

“These frequent episodes of mind wandering share a small-to-medium negative relationship with learning outcomes (Wong et al., 2022).”

This preserves the finding accurately without overclaiming causality.

I know this source I cited quite well and the answer the AI assistant gave me is correct. I could have phrased this better to avoid implying a causal impact. While this example is only with a few sentences and one source, the same type of prompt can be made with whole paragraphs, or even full drafts of manuscripts.

What happened behind the scenes is that the AI assistant searched my Zotero library for the source I cited . The Zotero MCP performed a semantic search and returned the matching entry, including the full text of the paper. The AI assistant then read the abstract and located the exact figure I cited: “students are off-task about 30% of the time during educationally relevant activities.” From there, it compared my phrasing against the original wording and identified two issues: first, the source uses “task-unrelated thought” rather than “mind wandering” as its primary term, and second, my phrasing implied a causal impact when the meta-analysis reported only a correlational relationship (r = -0.27). The key point here is that the AI assistant did not rely on its training data to verify my claims. It accessed the actual source stored in my Zotero library and checked my text against it.

Use Case 2: Internal Coherence

As academics we write a lot over the course of our career and might not always remember which stances we’ve taken or want to be consistent in the points we make across the papers we write. With our publications as part of the vault we can check whether what we’re arguing in a manuscript we’re working on aligns with the trajectory of work that we’ve already published. While it is of course okay to change our minds, to readers familiar with our work, this would need to be explained so they can also understand why we changed our minds. Let me give you an example with a conclusion section from a draft of mine:

I am working on the draft of a manuscript. Check whether this conclusion section I've written aligns with the arguments made in my previous publication, including work that is still under review. The section is:

Even among learners predisposed to reflect on their learning processes, the appraisal of whether a thought is aligned with their goals remains their own. Claiming that we can objectively appraise a learner's thoughts ignores the learners' agency in self-regulating their learning. If the learner is actively regulating their learning, they are the only ones capable of deciding whether a thought is congruent with their current goals or impedes their learning process. This transforms our role from external judges to facilitators who enable learners to make this appraisal.

This problem also extends to the literature on off-task thought detection using objective measures, such as eye-tracking (Kuvar et al., 2023). If these detection technologies were to be transferred to adaptive educational technologies such as the eye-mind reader by Mills et al. (2021), they risk not supporting learners' autonomous self-regulation of learning (Molenaar, 2022). By having a system pre-emptively make the decision for the learner, these technologies may absolve the learner of the need to develop their own meta-awareness and internalised refocusing skills. This could lead the learner to become reliant on the system rather than their own regulatory capacity. Therefore, any further study of mind wandering in a naturalistic setting needs to centralise the learner, normalise their experience of off-task thoughts, and enable a neutral appraisal of whether a thought is helpful to their specific learning trajectory. After all, it is their learning process.

Following from this prompt the AI assistant performed a lot of tool calls to search for and read my previous publications as well as all of my work that is under review and stored in my vault. Additionally, it performed multiple tool calls to access the sources I cited in my Zotero database and then provided me with the following answer:

Show AI response (Internal Coherence)

I’ve now reviewed the draft conclusion against your published and under-review work. Here is my assessment.

Overall Verdict

The conclusion is broadly consistent with the trajectory of your previous work, but it makes several extensions and shifts that need to be flagged. The most significant tension concerns your stance on objective detection measures (eye-tracking), which your thesis and conceptual paper treat as a complementary research tool, while this conclusion frames their deployment in adaptive educational technologies as potentially harmful to learner agency. The shift is defensible, but it is a new argument, not a restatement of prior positions.

1. Points of Strong Alignment

Learner agency and goal-dependent appraisal

The claim that “the appraisal of whether a thought is aligned with their goals remains their own” is well-grounded in your prior work:

Thesis conclusion (Ch. 8): “it is not the off-task thoughts that affect how well a learner is learning; it is the learner’s judgment concerning these off-task thoughts that impacts learning.”
Thesis (Ch. 8, future research): “the influence of off-task thoughts on learning should not only be studied in relation to goals set by the researcher, but also in relation to goals set by learners.”
Conceptual paper (Cognitive Science): The COPES framework positions the Evaluation of thoughts against learner-set Standards as the learner’s own metacognitive act.

The draft’s first paragraph faithfully extends this line of reasoning.

Normalising off-task thoughts

The call to “normalise their experience of off-task thoughts” aligns directly with:

Thesis (Final Remarks): “Instead of seeing off-task thought as the bane of successful learning… it is viewed… as something that is outside of the learners control and thus needs to be accepted.”
Conceptual paper: “It is normal for learners to experience TUTs, regardless of their self-regulation capabilities.”

Naturalistic setting as a research imperative

The draft’s call for studying mind wandering “in a naturalistic setting” is consistent with a core recommendation from the thesis: “research aiming to determine the relationship between SRL and off-task thoughts should be conducted in a naturalistic setting, as the transferability from experimental to naturalistic settings is not given.” The case study paper (Metacognition and Learning) was itself an argument for this.

Internalised refocusing

The concern about technologies that “absolve the learner of the need to develop their own… internalised refocusing skills” maps onto the thesis discussion of how self-regulated learners may “inherently refocus their thoughts back to the task through an internalised process whereby they are not explicitly aware of their off-task thoughts at all.” You are applying this concept to the technology question, which is a natural extension.

2. Points of Tension or Potential Misalignment

The stance on objective detection measures (eye-tracking)

This is the most significant area of tension. In your thesis and conceptual paper, eye-tracking and objective detection measures are framed positively as a research complement to self-caught thought reports:

Thesis (Ch. 8, implications for theory): “Such an approach to measuring off-task thoughts could be achieved by using objective modalities for off-task thought detection (for an overview see @kuvarDetectingWhenMind2023), in combination with self-caught thought reports.”
Thesis (Ch. 8, future research): Recommends “self-caught thought reports and objective measures of off-task thoughts [@kuvarDetectingWhenMind2023].”
Case study paper (Limitations): “This limitation could be overcome by including an objective measure of off-task thoughts that does not rely on the participants’ self-reports, such as eye-tracking (Kuvar et al., 2023).”

The draft conclusion, by contrast, frames the transfer of detection technologies to adaptive educational technologies as a risk to autonomous self-regulation. This is not a contradiction per se: your prior work discusses eye-tracking as a measurement tool for research, while the draft discusses its deployment in real-time intervention systems. However, a reader familiar with your thesis would notice that you previously advocated for objective measures and are now raising concerns about their application. The distinction between measurement for research and real-time intervention in adaptive systems should be made explicit in the manuscript to avoid the appearance of inconsistency.

The eye-mind reader (Mills et al., 2021) as a cautionary example

The draft cites the eye-mind reader as an example of a technology that risks undermining autonomous self-regulation. However, the literature note for this paper (@millsEyeMindReaderIntelligent2021) describes it positively: the system “promotes long-term comprehension by detecting and responding to mind wandering” and “led to an improved delayed learning outcome.” The system prompts self-explanation upon detection, which is itself a generative learning activity.

There is a tension here: the eye-mind reader does not simply “make the decision for the learner.” It detects mind wandering and then prompts the learner to engage in a self-explanation, which is a metacognitive activity. The system does not pre-emptively decide whether the thought was congruent with the learner’s goals; it intervenes after detection. Your critique would be more precisely targeted at systems that classify thoughts as detrimental or beneficial on the learner’s behalf without involving the learner in that appraisal. The eye-mind reader, as described, actually engages the learner in a response. You may want to either sharpen the critique or choose a different example.

“Neutral appraisal” language

The draft calls for enabling “a neutral appraisal of whether a thought is helpful to their specific learning trajectory.” The word “neutral” is slightly at odds with your COPES-based framing, in which the appraisal is inherently subjective: the learner evaluates thoughts against their own Standards. “Neutral” might be read as implying an objective stance, which is the very thing you are arguing against. Consider whether “learner-centred appraisal” or “self-directed appraisal” would be more consistent with the theoretical framework you have been building.

3. New Claims That Extend but Do Not Contradict Prior Work

These arguments are new in your oeuvre. They are logical extensions but have not been substantiated in your prior publications:

“This transforms our role from external judges to facilitators” — The thesis discusses implications for practice in terms of scaffolding SRL skills and teaching reaction tactics, but does not frame the researcher’s or educator’s role as “facilitator” versus “external judge.” This is a new rhetorical positioning.
“These technologies may absolve the learner of the need to develop their own meta-awareness” — This is a novel claim. Your prior work discusses meta-awareness as a skill that learners develop through metacognitive monitoring cycles, but you have not previously argued that external detection systems could displace the development of this skill. The Molenaar (2022) HHAIR concept supports this argument (ALTs “take over (offload) regulation from learners”), but you are applying it to mind wandering detection specifically, which Molenaar does not address.
“This could lead the learner to become reliant on the system rather than their own regulatory capacity” — This is a dependency argument that is not made in your prior work. It is plausible and consistent with the HHAIR framework, but it is an empirical claim that would benefit from being framed as a hypothesis rather than a stated consequence.

4. Citation and Reference Concerns

Molenaar (2022): This refers to “The concept of hybrid human-AI regulation” (Computers and Education: Artificial Intelligence, Vol. 3, 100070). It is in your Zotero library but has no literature note in the vault. The citation is appropriate for the argument about autonomous self-regulation, as Molenaar explicitly argues that ALTs offload regulation from learners. However, note that Molenaar’s concept is about gradual transfer of regulation from AI to human, not about abandoning AI support entirely. Your draft’s argument is stronger than Molenaar’s position: where she advocates for hybrid regulation with gradual transfer, you seem to caution against the system making the decision at all. Make sure you are not overextending Molenaar’s claim.
Mills et al. (2021): As noted above, the eye-mind reader prompts self-explanation rather than making a pre-emptive decision for the learner. The characterisation in your draft may misrepresent the system’s actual function.
Kuvar et al. (2023): This is correctly cited as a systematic review of mind wandering detection methods, consistent with how it is cited in your thesis and conceptual paper.

Summary of Recommendations

Clarify the measurement vs. intervention distinction. Your prior work endorses eye-tracking as a research measurement tool. The draft critiques its deployment in adaptive intervention systems. Make this distinction explicit so readers do not perceive inconsistency with your thesis.
Sharpen or replace the eye-mind reader example. The system prompts self-explanation, which engages the learner. If your critique targets systems that pre-emptively classify thoughts without learner involvement, the eye-mind reader is not the strongest example. Alternatively, reframe the critique to acknowledge that the eye-mind reader engages the learner but still bypasses the learner’s own detection of the thought.
Reconsider “neutral appraisal.” Your COPES framework positions appraisal as the learner’s subjective evaluation against their own Standards. “Neutral” may undercut this.
Flag the new claims as hypotheses, not established consequences. The arguments about absolving learners of meta-awareness development and creating system dependency are extensions of your prior work, not substantiated by it. Framing them as risks or hypotheses would be more consistent with the calibrated hedging in your other publications.
Consider creating a literature note for Molenaar (2022) in the vault, given that you are now citing it in manuscript drafts.

This answer is a good example of how the PKM enabled the AI assistant to give a useful answer. I stored the full text of all my work in my vault and, therefore, the AI was able to access these and use them to inform its answer to me. For example, when we look at section 1 of its answer on points of strong alignment we can see how it is pulling from a chapter of my PhD thesis as well as a paper of mine that was recently published to then conclude that the reasoning in the section I gave it aligns with the reasoning in my thesis and the recent paper. Not only that, it also correctly identifies that there are points of tension where my thinking has evolved in regard to automated mind wandering detection technologies. While I don’t necessarily agree with all of the content of the AI’s answer, it clearly shows that this answer is strongly informed by the data sources provided to it and the points it makes give me something to think about that might result in me changing something in my writing.

Separately from the quality of the answer, this is also a good example of AI not always strictly following instructions. In the prompt I had asked it to check this section against my previous publications and works that are still under review but already stored in my vault. Further, as previously shown, in my agent definition I’m instructing the AI assistant to not be proactive and to closely stick to the instructions given. But, despite all of that, we can see that the AI assistant still ran checks for each source I am citing if the source supports the claim I am making, even though I did not instruct it to do so and did not want it to be proactive. This is a constant point of frustration, but there is only so much we can do to control the AI assistant’s behaviour.

Use Case 3: Learning from Reviews

The last use case I would like to present here is related to paper submissions and reviews. When we submit a paper to a journal that usually gets reviewed by some of our peers and only if our peers and the editor deem it good enough will it be published. As part of each review we get peer-review reports or feedback. I store every single review I’ve gotten since I started my PhD in my vault and that presents a learning opportunity, but once again, we as humans forget. However, an AI with access to all of these reviews can read a draft before we submit it to a journal and then go over the reviews and determine which points previous reviewers made might also apply to the current draft I am working on. As with the previous two use cases, let me give you an example. As the draft for this example I’m using the .

I am working on a draft of a paper. Check this draft located in the vault at "Publications/Experiment Interpolated Testing vs Self-Explanation/Meta-Awareness of Mind Wandering (JCAL)" against all previous peer-reviews I've received and determine which critiques from previous peer-reviews might also apply to this work along with a suggestion for how I could improve this work to avoid getting the same critique.

After the AI assistant read my draft and my previous peer-reviews, the answer I got was:

Show AI response (Learning from Reviews)

I now have a comprehensive picture of all peer reviews Daniel has received across his publications and the current JCAL v2 draft. Let me compile the analysis.

Cross-Review Analysis: Recurring Critiques Applied to the JCAL v2 Draft

I have reviewed the v2 draft against all peer reviews you have received across the following submissions:

Mind Wandering Reactions Case Study (LAK25, then Metacognition and Learning, rounds 1 and 2)
SRL and Mind Wandering Conceptual Paper (Metacognition and Learning, then Cognitive Science)
TRI Meta-Analysis (Review of Educational Research rounds 1 and 2, Psychonomic Bulletin & Review, Contemporary Educational Psychology)
SmartQuiz Experiment (AIED 2026)
ECTEL 2024 Doctoral Consortium
LAK 2024 Doctoral Consortium
HERGA 2023 Poster

Below are the critiques from these previous reviews that also apply to the current JCAL v2 draft, ordered by severity (likelihood of recurrence and impact). Each entry identifies the source review(s), the specific issue in the v2 draft, and a concrete suggestion.

1. Theoretical model-measurement misalignment (COPES vs. self-report SRL-O)

Source reviews: LAK25 meta-review (Reviewer 1 + meta-reviewer); Metacognition and Learning Reviewer 2 (Comment 10).

Previous critique: The COPES model (Winne) conceptualises SRL as a series of dynamic, task-level events. Winne has advocated for decades for trace-based measures. Using a self-report aptitude questionnaire (SRL-O) to operationalise COPES constructs is a fundamental theory-measurement mismatch. The LAK25 meta-reviewer stated explicitly that this cannot be resolved by arguing self-reports are acceptable for meta-awareness; the entire theoretical model would need to be changed (e.g., to Zimmerman’s) or the measurement approach changed to traces.

How it applies to the JCAL v2: Section 2.6 draws on the COPES model as the theoretical framework, and Section 4.4 uses the SRL-O self-report questionnaire to measure SRL aptitude. The same mismatch exists. The JCAL v2 does not address or even acknowledge this tension. A reviewer familiar with Winne’s work (or the LAK25 meta-reviewer, if they review for JCAL) would flag this immediately.

Suggestion: Add an explicit justification for why a self-report aptitude measure is acceptable within the COPES framework for this specific research question. The argument should be that RQ2 examines the relationship between SRL aptitude (a trait-like characteristic) and meta-awareness frequency (also trait-adjacent), not SRL events. Alternatively, reframe the theoretical framework to use a model that is more compatible with aptitude measures (e.g., Zimmerman’s three-phase model or Pintrich’s framework), or acknowledge the mismatch in the limitations section. Do not leave it unaddressed; the LAK25 meta-reviewer considered this a fatal flaw.

2. No reliability estimates reported for SRL-O subscales in the current sample

Source review: Metacognition and Learning Reviewer 3 (Comment 23).

Previous critique: The reviewer noted that the manuscript reported external Cronbach’s alpha benchmarks (0.76-0.88 from prior studies) but placed the current sample’s reliability estimates in a supplementary table. The reviewer called this “methodologically problematic” because reliability is sample-dependent and external benchmarks should not substitute for transparent reporting in the present dataset.

How it applies to the JCAL v2: The v2 draft (Section 4.4) describes the SRL-O subscales used and cites Broadbent et al. (2022) but does not report Cronbach’s alpha (or equivalent) for the Metacognition, Online Effort Regulation, or Online Task Strategies subscales in the current sample of 138 participants. This is exactly the same omission that was previously flagged.

Suggestion: Report internal consistency estimates (Cronbach’s alpha or McDonald’s omega) for each SRL-O subscale in the current sample, either in the Procedure section or in a table. This is a straightforward addition that pre-empts the critique entirely.

3. Causal language for correlational findings

Source reviews: Metacognition and Learning Reviewer 3 (Comment 25: “conflates correlation with causation and overgeneralizes from the findings”); RER Reviewer 1 (Comment 15: “causal language is not warranted given the non-experimental nature of the data”); Psychonomic Bulletin & Review Reviewer 3 (Point 4: “educational implications overgeneralize from lab studies”).

How it applies to the JCAL v2: Several passages use directional or causal language for correlational findings:

Abstract: “indicating that meta-aware learners perform better on subsequent knowledge tests” implies that meta-awareness causes better performance.
Lay summary: “Students who reported being better at self-regulating their learning reported fewer unrelated thoughts” (this one is correctly hedged).
Conclusion: “only if learners are unaware of their off-task thoughts does it negatively affect learning” implies causation from a correlational result.
Discussion (Section 6): “an increase on the scale corresponded to a decrease in the number of thought reports” is acceptable, but “meta-aware learners perform better” is not.

RQ2 and RQ3 are correlational/exploratory, not experimental. RQ1 is experimental but produced null results. The positive correlation between thought reports and knowledge gain (tau = .15) is a weak effect and should not be interpreted as evidence that meta-awareness improves learning.

Suggestion: Audit the abstract, discussion, and conclusion for causal language. Replace “meta-aware learners perform better” with “meta-awareness was positively associated with knowledge test performance.” Replace “only if learners are unaware… does it negatively affect learning” with “the negative association between off-task thoughts and learning may be limited to instances where learners lack meta-awareness.” Add a sentence in the discussion explicitly noting the correlational nature of RQ2 and RQ3 findings and that no causal claims can be made.

4. “So what” / practical implications underdeveloped

Source reviews: JCAL Reviewer 2 (Comment 16: “biggest weakness is the lack of clear answers to the ‘so what’ question”); Cognitive Science Reviewer 1 (Comment 4: “insufficient discussion of how educators might leverage this understanding”); RER Reviewer 1 (Comment 16: “I’d like to see more recommendations for practice, and not just future research”).

How it applies to the JCAL v2: The practical implications paragraph in Section 7 is brief and essentially says: these approaches do not work for short videos, so consider video length, and encourage note-taking. The conclusion focuses on theoretical implications but the practical takeaway is thin. The main novel finding (positive correlation between thought reports and learning) is not translated into a concrete actionable recommendation. The lay summary says “Teaching students awareness of unrelated thoughts could result in more efficient learning sessions” but this is not elaborated in the discussion.

Suggestion: Expand the practical implications to address: (a) what should educators do given that neither interpolated testing nor self-explanation improved outcomes for short videos? (b) what does the positive correlation between meta-awareness and learning suggest for instructional design? (c) what specific strategies could be used to foster meta-awareness (e.g., brief reflection prompts, awareness training)? Connect these to the Randall et al. (2022) reference already cited, which studied self-regulatory strategies to reduce mind wandering.

5. Inter-rater reliability (kappa = 0.47) is moderate and could be questioned

Source reviews: RER Reviewer 1 (Comment 10: “Cohen’s Kappa values were not particularly high”); RER Reviewer 2 (Comment 3: “low inter-rater-reliability has implications for the overall reliability of the screening process”).

How it applies to the JCAL v2: Section 4.5 reports an initial Cohen’s kappa of 0.47, which is moderate agreement. The draft states that all disagreements were discussed until 100% agreement was reached. However, a reviewer could question whether the initial moderate agreement indicates that the coding scheme itself is ambiguous, and whether the resolution process (discussion to consensus) introduced coder drift or bias. This is especially relevant because the thought type categories (TUT, TRI, ED, on-task) involve subjective judgment.

Suggestion: Add a brief justification for the moderate kappa, noting that thought report coding involves inherent subjectivity in interpreting free-text responses. Report the number of thought reports coded and the number of disagreements. Consider reporting kappa after the first round of discussion (not just the initial value) to show improvement. If possible, report a final kappa on a held-out subset to demonstrate post-consensus reliability.

6. TRI category conflates metacognitive monitoring with interference

Source reviews: RER Reviewer 2 (Comment 1: “TRI definition has a large overlap with metacognition”); Psychonomic Bulletin & Review Reviewer 3 (Point 1: “the distinction between TRI vs. metacognition seems blurred”); Contemporary Educational Psychology Reviewer 1 (Major Concern 1: “Labeling such processes as ‘interference’ is problematic. It conflates potentially beneficial cognitive activities with genuinely distracting thoughts”).

How it applies to the JCAL v2: Section 4.5 defines TRI as “a thought about the task that represented an interruption in the flow of the task (e.g., thinking about the task’s difficulty, questioning understanding).” “Questioning understanding” is a metacognitive monitoring judgement, not interference. This is the exact conflation that was flagged across three separate submissions of the TRI meta-analysis. A reviewer who has encountered this critique before (or who is familiar with the metacognition literature) would flag it.

Suggestion: Refine the TRI coding definition to distinguish between metacognitive monitoring (which is on-task and beneficial) and genuine task-related interference (thoughts that interrupt the flow without contributing to learning). Consider whether “questioning understanding” should be coded as on-task rather than TRI. Add a sentence acknowledging the conceptual overlap and explaining the coding decision.

7. Data exclusion and outlier handling

Source reviews: AIED 2026 (Review 2 and 4: “excluding participants based on outcome variables risks introducing bias”; “exclusions appear uneven across conditions”); RER Reviewer 1 (Comment 12: “more information on outlier criteria”); Contemporary Educational Psychology Reviewer 2 (Point 5: “outlier detection should be based on the model, not descriptive evaluation”).

How it applies to the JCAL v2: Section 4.7 excludes 42 participants for incomplete data, then applies outlier detection using “the standard deviation method, a cutoff of two standard deviations, and the interquartile range method” simultaneously, excluding all identified outliers. The resulting group sizes are uneven (47, 51, 40). The use of multiple simultaneous outlier criteria is unusually conservative and the rationale is not provided. The uneven group distribution after exclusion could introduce bias.

Suggestion: Justify the use of multiple outlier detection methods. Report how many participants were identified as outliers by each method and how many were excluded in total due to outliers (as distinct from the 42 who did not complete the study). Consider reporting sensitivity analyses with and without outliers to show robustness. Address the uneven group sizes in the limitations.

8. No figures for results presentation

Source reviews: Metacognition and Learning Reviewer 1 (Comment 8: “provide at least 1 or 2 figures for your primary results rather than 10+ large tables of stats”); Psychonomic Bulletin & Review Reviewer 1 (Point 4: “forest plots should be in main text, not supplementary”).

How it applies to the JCAL v2: The results section (Section 5) presents all findings in tables (Tables 1-5). Figure 1 is a procedure diagram. There are no figures visualising the primary results (e.g., group comparisons, SRL-thought report relationships, correlation scatter plots). This was previously flagged as a readability issue.

Suggestion: Add at least one figure visualising the primary result of interest. For RQ1, a bar plot of thought reports by group with error bars. For RQ2, a coefficient plot showing the GLM estimates for each SRL-O subscale. For RQ3, a scatter plot of thought reports vs. knowledge gain. This improves readability for a journal audience and pre-empts the critique.

9. Terminology density / jargon in abstract and introduction

Source reviews: Metacognition and Learning Reviewer 1 (Comment 1: “results discussed in the abstract were quite jargony”); Metacognition and Learning Reviewer 3 (Comment 20: “presupposes reader familiarity with domain-specific terminology”); ECTEL 2024 DC Reviewer 2: “significant overlap of complex SRL terms, but they are not all well-addressed.”

How it applies to the JCAL v2: The abstract uses terms like “interpolated testing,” “self-caught thought reports,” “meta-awareness,” “metacognitive monitoring,” and “online task strategies subscales” without definition. The lay summary is more accessible, but the structured abstract itself remains jargon-heavy. JCAL has an interdisciplinary audience (education + technology), not all of whom will be familiar with mind wandering terminology.

Suggestion: Add brief parenthetical definitions for key terms in the abstract (e.g., “self-caught thought reports (participants self-report off-task thoughts as they notice them)”). Ensure that the first use of each acronym in the introduction is accompanied by a definition. The lay summary already helps here, but the structured abstract should be independently accessible.

10. Individual differences in meta-awareness ability not addressed

Source review: Cognitive Science Reviewer 1 (Comment 2: “the model assumes relatively uniform ability to detect TUTs; needs to address how working memory capacity, executive function, and attention control might moderate the SRL-TUT relationship”).

How it applies to the JCAL v2: The JCAL draft assumes that all participants have equivalent capacity to become meta-aware of their off-task thoughts. The finding that Metacognition subscale scores negatively predict thought reports (the opposite direction of what the scale ostensibly measures) could partly be explained by individual differences in meta-awareness ability. The draft discusses this in terms of “metacognition without awareness” (Reder, 1996) but does not address stable individual differences in the capacity for meta-awareness.

Suggestion: Add a brief discussion in the limitations or discussion section noting that individuals vary in their capacity for meta-awareness (citing Grabman & Dodson, 2024, which is already in the Cognitive Science version). Acknowledge that this variability could partly explain the unexpected direction of the Metacognition subscale results. This is a low-effort addition that pre-empts a critique you have already received.

11. Video duration and ecological validity

Source reviews: JCAL Reviewer 2 (Comment 11: “intervention was very brief… inappropriate to expect these skills to change”); Psychonomic Bulletin & Review Reviewer 3 (Point 4: “educational implications overgeneralize from lab studies”); JCAL Reviewer 1: “more control of variables such as video content.”

How it applies to the JCAL v2: This was already raised by JCAL’s own Reviewer 2 in round 1 and is partially addressed in the limitations (Section 7). However, the practical implications paragraph still generalises from a single 8-minute video to recommendations for educators. The null findings for RQ1 could be entirely attributable to the video being too short for mind wandering to accumulate, which undermines the practical recommendation that interpolated testing and self-explanation are ineffective.

Suggestion: More carefully qualify the practical recommendations to specify that they apply to short videos only. State explicitly that the 8-minute video limits the generalisability of the null RQ1 findings and that the effectiveness of these interventions may differ for longer videos. This is already partially present but should be tightened.

Summary Table

#	Critique	Source Review(s)	Severity
1	COPES-measurement misalignment	LAK25, M&L R2	Critical
2	No reliability estimates for SRL-O	M&L R3	High
3	Causal language for correlations	M&L R3, RER R1, PBR R3	High
4	“So what” / practical implications	JCAL R2, CogSci R1, RER R1	High
5	Moderate inter-rater kappa	RER R1, RER R2	Medium
6	TRI/metacognition conflation	RER R2, PBR R3, CEP R1	Medium
7	Data exclusion / outlier handling	AIED 2026, RER R1, CEP R2	Medium
8	No results figures	M&L R1, PBR R1	Medium
9	Jargon in abstract	M&L R1, M&L R3, ECTEL DC	Low-Medium
10	Individual differences not addressed	CogSci R1	Low-Medium
11	Video duration / ecological validity	JCAL R2, PBR R3, JCAL R1	Low (partially addressed)

The most critical item is #1 (COPES-measurement misalignment). This was considered a fatal flaw by the LAK25 meta-reviewer and it remains unaddressed in the JCAL v2. I would prioritise addressing this before the next review round, as a reviewer familiar with Winne’s work will almost certainly raise it again.

After reading this review, we can clearly see that the AI assistant systematically processed the entire manuscript I gave it (about 9000 words), then went through each peer-review I’ve gotten and outlined in detail with reference to the original review how the point made by previous reviewers would apply to the current manuscript I’m working on and how I could address it preemptively. While I might not agree with every point raised or think that some other points are not as severe as the AI makes them out to be, these do give me valuable feedback on what I can address in my manuscript to improve my work and increase my chances of acceptance.

What enabled the AI assistant to do this was the way my reviews are stored in the vault. Each review is stored as a note with the metadata type: peer-review and filed in the same folder as the publication it relates to. That means the AI assistant could filter for all notes with that metadata type and then read through them alongside the current draft. For example, when the AI assistant flagged the COPES-measurement misalignment as the most critical issue (point 1 in its output), it was pulling from the LAK25 meta-review stored in my vault, which had explicitly called this a fatal flaw. It then matched that critique against my current draft, in which I use the COPES model as a theoretical framework. The same applies to point 3 (causal language for correlations), which traces back to critiques I received from three separate reviewers across three different submissions, all stored in the vault. The AI assistant was able to cross-reference all of these against the current draft in a single pass. Without the reviews being stored in a structured, machine-readable way, this kind of cross-referencing would not have been possible.

Based on these use cases and examples I hope you can now understand why I consider this specific setup so powerful and how helpful it can be.

Temptations to Resist

Beyond these practical applications, there are broader considerations worth addressing. Now that you’ve gotten a glimpse of the type of usage this type of system can be put to, it is time to address the elephant in the room called responsible usage of AI. and what is considered responsible or acceptable use of AI will differ based on who is asked and what the AI is being used for in which context. It is a wide range between asking an AI to explain something, which most people would be fine with, to asking an AI to generate a full paper for you, which I think most people would object to.

I am firmly of the position that human oversight of AI-generated content is essential, which is also reflected in the language used in these posts as I refer to an AI assistant. To stick with the metaphor, an assistant does not do work by themselves; an assistant assists you, as the word itself suggests. That means you are in charge and responsible. I of course realise that this boundary is all too easy to cross and any technology can be appropriated and used for a different purpose, but it also comes with an intent, and the intent of this specific combination of technology I’ve been writing about is only to assist you, and I trust your moral judgement to draw the line between using the AI to assist you and the AI doing the work for you.

A helpful guideline is to not use the AI for tasks that you couldn’t do yourself. If you couldn’t do the task yourself, you’re lacking the skills and knowledge to assess the AI’s output and thus shouldn’t be using it for this case. Furthermore, another helpful guideline is to consider the distribution of intellectual labour between yourself and the AI. The intellectual contribution of the work should be yours, and that is exactly the point being objected to when someone uses AI to generate a full paper. But, what if the AI is generating something based on your work, as this setup enables, and under your instruction and revision? This is a fine line and a judgement call you will have to make for yourself.

Next Steps

Now that we’ve covered my entire current setup along with some use cases it is time for you to get to work on your own setup. I recommend you start by analysing which software you’re already using as part of your workflow and which state the sources of information and knowledge you rely on are in. Let’s say you already have an Obsidian vault but it doesn’t contain your publications in full text then a next step could be to convert those to markdown and import them, or you might already have both Obsidian and Zotero in place and have the information in those highly organised, then the next step would be to set up the MCPs and get started on using this system. The starting point will be very individual but hopefully the explanations given in this series have enabled you to make an informed decision about how to set up a system that works for you. One practical note: for the MCPs to work, both Zotero and Obsidian need to be running when you use the AI assistant, as the MCPs connect to them via their local APIs.

Future Development

While my current setup is already very useful to me, there always are things that could be improved. A few things that I’m considering to try are things like (a library that gives agents persistent memory across sessions). Currently each session starts from scratch, or in other words, the AI assistant does not retain information between sessions. Enabling some sort of memory across sessions would get rid of this limitation and potentially prove useful down the line. Other avenues worth exploring are to make the setup multimodal. Currently it is primarily text based, simply because most of my work is text based, but sometimes being able to generate images would be useful, for this something like the (which enables image generation through the same router you are already using) could be useful. Another potentially useful addition would be to enable the creation of office documents like Word files or PowerPoint presentations, and while , I am not sure if they are needed if . When considering the next steps in this journey something like the project (an open-source framework for building personal AI assistants) is useful inspiration, although I will likely build my own system as I want to continue to be able to exchange each part, as explained in this series, when needed.

Closing Note

As a final note, I should mention that I used the AI assistant described in this series to refine these posts. From checking internal coherence to verifying that claims are supported and terminology is used consistently, the system fed information about itself into posts about it.

If you’ve followed along this whole series of post and think it would be useful to attend a workshop on how to build your own AI research assistant please , I will gladly give such a workshop.

AI Research Assistant: Agent Definition (4/5)

Fri, 24 Jul 2026 00:00:00 +0000

If you followed this series so far, you will now have a runtime in place that you can use for your AI assistant setup. That means the next steps are to define your environment to the AI and to define the AI assistant’s personality, which we will cover in this post. For this purpose we will cover what an AGENTS.md file is and how to write it, and how to structure the agent definition for your AI agent. The difference between the AGENTS.md file and an agent definition is that the AGENTS.md file contains information about the folder (in this case your Obsidian vault) that is relevant for all AI assistants while the agent definition covers the role and personality of those AI assistants, as I will outline in more detail below. Lastly, I’ll cover how all the parts we’ve been working on throughout this series then go together when you prompt the AI assistant to do some kind of task for you.

AGENTS.md

Think of an file as a README file for agents, or in other words, a file that tells agents how to interact with something in a folder, what this folder is about, and what rules to respect. This file is located at the root of your vault.

In this file you describe to the AI assistant what the environment is in which it works. This includes things like stating that the folder is an Obsidian markdown vault with a connection to Zotero, but also which platform it is running on (Windows, Mac, Linux, or so), which shell is available (PowerShell, bash, zsh, fish, or so). Further, it includes things like hard rules on which commands are allowed and which are forbidden, for example, always forbidding git commit commands without being told to commit. Other things to cover in this file are what are key directories for you and what the naming conventions are in these files. I realise that this is quite abstract, therefore, here is an example.

# AGENTS.md - Research Vault

This is an Obsidian-flavored Markdown knowledge base, not a software project. There is no build, lint, test, or deploy pipeline.

## Environment
- **Shell:** Windows PowerShell.
- **Python:** `.\.venv\Scripts\python` only. Never system `python`.

## Hard Rules
1. **Never `git commit`** without explicit instruction. Read-only git commands (`status`, `diff`, `log`) are fine.
2. **Never create literature notes manually.** New papers go into Zotero first, then import via the Zotero Desktop Connector.
3. **No em-dashes (—) in any text.** Zero exceptions. This is the strongest AI-writing tell.
4. **Wikilinks only** (`[[Note Name]]`), never filesystem paths.
5. **UK/AU spelling** by default.

## File Conventions
- **Literature notes:** `@citekeyFirstAuthorYYYY.md` in `Literature/`.
- **Tags:** `lowercase` or `camelCase`, no spaces, no kebab-case. E.g., `#mindWandering`, `#selfRegulatedLearning`.
- **Dates:** ISO 8601 (`YYYY-MM-DD`).
- **Frontmatter:** Every `.md` file requires valid YAML frontmatter.

## Key Directories
| Directory | Purpose |
|-----------|---------|
| `Literature/` | Bibliographic notes, Zotero-imported only |
| `Publications/` | Full-text manuscripts and drafts |
| `WIP/` | Active projects |

## Research Context
Post-PhD research in Self-Regulated Learning (SRL), mind wandering, and educational technology.

What this example AGENTS.md contains is first a high level overview of the project. In this case stating that our folder is an Obsidian research vault. Importantly, interfaces such as OpenCode are built to be used in the context of software development and thus we’re specifically stating that this is not the case here to avoid the AI assistant treating this project as such. The overview is followed by a specification of the environment. I am reluctantly using Windows and thus the AI assistant will use PowerShell. This will need to be adjusted based on your system, for example, if you’re on macOS the default shell is zsh. If you’re on Linux, this will depend on your distribution, but telling your AI assistant to use bash should work. Further, I specify that the AI assistant should maintain its own virtual environment for python to avoid globally installing dependencies.

The hard rules section is important as AI assistants have the tendency to take actions that we might not want. For example, I explicitly forbid AI assistants from committing anything unless I instruct it to do so. Without that line my experience was that they would commit after every small task I gave them. After the hard rules I define file conventions and key directories, such as where literature notes are stored. These will very much depend on how you like to structure your work (see ). When writing your own AGENTS.md, adjust it to the structure of your notes. Lastly, I provide a short notice about the research context in which I’m working.

One thing I want to warn against is overfilling your AGENTS.md. A bloated AGENTS.md file can lead to AI assistants being less useful while increasing the cost as every line goes into every session and can thus misdirect the AI assistant’s focus. Do not include detailed directory listings the AI assistant can discover on its own, style guidelines, or task-specific instructions that only apply sometimes.

Agent Definition

Once the AGENTS.md is in place the next step is to define the AI assistant you want to interact with. Unlike the AGENTS.md file, how AI assistants are defined is not standardised and will depend on the interface you’re using. For the following I assume you’re using OpenCode like I am. that are located in the folder .opencode/agents. Importantly, use the .opencode/agents/ folder within your Obsidian vault, as agents can be defined on a global level but also on a folder level, and in this case we only want to define this AI assistant on a folder level. This can be called whatever you like, for example, research-assistant.md. It is in this file that you define what role you want your specific AI assistant to fill. When defining the AI assistant, there are two parts that need to be written, the yaml frontmatter and the markdown text outlining its behaviour. The key principle here is: do not duplicate. If every AI assistant that might ever be pointed at your folder requires it, put it in AGENTS.md once. If it is specific to this AI assistant’s role or voice, put it in the agent definition. A simple test: if you swapped in a completely different AI assistant persona tomorrow, would this instruction still need to hold? If yes, it belongs in AGENTS.md.

When defining your AI assistant think about what this specific AI assistant should do in terms of its role, the style used, common tasks, but also boundaries it can’t cross. What we have been working towards, and what I am using in my setup, is an AI assistant that takes on the role of a research assistant. For this role I specify a high-level description in the yaml frontmatter and in the markdown portion of the agent definition file I outline in more detail what I want my AI assistant to do. Key points to cover in this section are the AI assistant’s role, information about the user, operational principles such as to not be sycophantic, and important theoretical positions relevant to your work. As with the AGENTS.md file above, I realise this is quite abstract, as such I also provide an example and will further explain this example below.

---
description: Research assistant for managing academic workflows, literature management, and vault synthesis.
mode: primary
color: "#5E81AC"
permission:
 webfetch: allow
 edit: allow
 skill: allow
 read: allow
---

# Research Assistant

You are the **research assistant** for an academic research vault. Your role is to assist with literature discovery, knowledge synthesis, analysis and critique of my writing, and responding to peer reviews.

## User Context
- **Name**: Daniel Ebbert
- **Role**: Research Fellow
- **Affiliation**: Adelaide University
- **Primary Focus**: Postdoc research in self-regulated learning, mind wandering, and educational technology.

## Values & Behaviour

### Personality Traits
- **Non-Sycophantic**: Do not blindly agree with the user. Actively question assumptions and provide critical feedback.
- **Disciplined**: Follow instructions precisely. Do not presume to know what the user wants next. You are NOT proactive.
- **Academic Tone**: Maintain a professional, objective, and scholarly voice.

### Key Beliefs & Perspectives
- **Theoretical stance**: Do not rely on or refer to the "family resemblance" framework of mind wandering (Seli et al., 2018).

## Academic Voice (when drafting text)
- **Calibrated hedging**: confident about prior literature ("demonstrates"), cautious about own findings ("suggests", "indicates").
- **p-value interpretation**: interpret p-values strictly against the alpha threshold. There is no such thing as "marginally significant."

## Zotero Operations
- Prefer semantic search over keyword search for topic queries; it is more efficient.

## Subagents
You have one subagent, literature-search. Call it when you search for academic literature beyond what is included in Zotero.

The above example first contains the yaml frontmatter that OpenCode expects in which I define that this AI assistant is a research assistant and a primary agent (the other type are subagents which I explain in the following section). The permission block is where you scope what this AI assistant can and cannot do. The point is not to grant everything but to restrict. A research assistant that only reads and synthesises literature does not need permission to run arbitrary shell commands. Tool restrictions prevent destructive mistakes and keep the AI assistant on task.

Following the yaml frontmatter is the role description again, this time in a bit more detail. After the role I provide the AI assistant with some information about me, such as who I am and what I primarily research. With those points out of the way, the next step is to define values and behaviours for the AI assistants. Key points that I want to highlight here are the personality traits. AI assistants have the tendency to be sycophantic and while that is great for polishing the ego of some users, it does not serve our purpose in the context of doing research, therefore, I am here instructing the AI assistant to be non-sycophantic. The next point is a matter of personal taste. I personally don’t like it when the AI assistant is proactive and takes actions based on what it infers I want it to do. In my opinion that too often goes wrong and then you will only have wasted time and tokens, and tokens mean money. Therefore, I instruct my AI assistant to never do anything that I didn’t instruct it to do. Although I realise that some of you will want to deviate from this point.

Another important point that I want to highlight is that of key beliefs and perspectives. This AI assistant will have access to all our notes and literature. What it doesn’t account for is that we sometimes read work that we (strongly) disagree with and would not base our work on. If that is the case and we want to avoid the AI assistant drawing on that literature then we need to state that. For example, I do mind wandering research and there is this theory of the family resemblance framework of mind wandering that I disagree with. Therefore, I instruct my AI assistant not to rely on or refer to it. Lastly, there is the point of subagents, as each subagent defined should be listed in the agent definition along with a description of what to call each subagent for.

To wrap up the agents definition section, keep in mind that you do not need to build everything at once. Everything you’ve read about in this blog series might seem purpose-built, but it was not, it has grown through experience and so should your setup, including your agent definition.

Subagents

In the spirit of transparency, let me start by stating that the concept of subagents is still very much a work in progress in my setup. I am aware that subagents can be used and have tried that a little but the subagents setup is not as mature as the rest of my setup nor am I writing with the same level of knowledge and confidence on the topic as in the rest of this series.

AI assistants can be divided into primary agents that you directly interact with, such as the AI assistant defined above, and that are called by primary agents. Think of subagents as AI agents with a narrowly defined scope and task. They are only enacted when their task is needed, and multiple can be called at the same time. Some interfaces will already have built-in subagents. For example, OpenCode has a built-in subagent called explore. This subagent is used to explore the files in a folder and a primary agent can call this subagent multiple times for this purpose and these subagents then report back to the primary agent. Another important feature of subagents is that they can run in parallel. That means the primary agent can call subagents to fulfil a task while at the same time working on something else.

In the context of the project this series of posts is about, the only useful subagent I have defined is for literature search. That means in practical terms that my AI assistant can hand off the task of searching for new literature to the subagent while in the meantime searching through my vault and Zotero database for the same information.

Subagents are defined in the same way as primary agents, with the small difference that the mode variable in the frontmatter gets the value subagent instead of primary. This could look like the following example.

---
description: Global literature discovery via OpenAlex structured search, citation mapping, and preprint scouting.
mode: subagent
permission:
 read: allow
 webfetch: allow
 edit: deny
 bash: deny
---

# Literature Search

You are a **literature discovery specialist**. Your role is to find academic papers beyond what is stored in the user's Zotero library.

## When to use this AI assistant
- The user asks for papers on a topic not covered by existing Zotero entries.
- The user needs citation graph traversal or preprint scouting.
- The main agent needs to discover work outside the curated library.

## Scope
- Use OpenAlex for structured searches.
- Return metadata, abstracts, and citation counts.
- Do not add items to Zotero. Only the user adds papers.
- Do not modify vault notes. Return results to the main agent.

The structure of this file is the same as for primary agents, although in this case the description of the AI assistant is of more importance as the primary agent has access to the subagent description and uses it to determine whether to invoke a subagent or not. The main agent sees only the description field; the full body loads only when the subagent is invoked.

Small side note here, my subagent is defined to use OpenAlex as it is a database with a very wide coverage and an API that lends itself nicely to be queried by an AI. If you’re not familiar with , I encourage you to check it out.

One last point on the structure of agents and subagents: it is flexible. In my setup, I have one primary agent in the role of a research assistant with a relatively general description, and one subagent for literature search. That is not the only option. I could have defined multiple smaller-scale primary agents instead, for example one for knowledge retrieval and synthesis in my vault and one to act as . Similarly, I could have defined separate subagents for interacting with Zotero and one for interacting with Obsidian. Or I could have kept the research assistant as the primary agent and added subagents for knowledge retrieval and synthesis, and one as reviewer 2. The point is that the division between primary agents and subagents is a design choice you have to make. There is no single correct architecture nor can I make a recommendation for which architecture is useful for which case. The best structure is the one that fits your workflow, which you will discover through use. Therefore, I recommend that you start small with one simple primary agent and only add complexity when needed.

How It All Goes Together in a Prompt

The context stack loaded into every session, in order.

With the agent definition done the system I’ve been writing about is ready to use, but before doing that, I would like you to understand what goes into each session, in every new chat. Every time you start a new chat a lot of information is loaded into the context and processed by the AI model before it even reaches your prompt. Understanding what goes into each session will help you develop a mental model useful in adjusting your AI assistant setup. The following goes into each session:

The developer system prompt. These are fixed instructions from the developers (and possibly the providers). Often these are safety policies, such as when . We as the users usually don’t have access to the prompts at this layer, but some examples are available.
The interface system prompt. These are instructions from the interface you’re using, such as OpenCode or Claude Code, on how to format tool calls or how to handle session states and how to structure conversations.
Your AGENTS.md file. The AGENTS.md file you defined, as outlined above.
The agent definition. The definition for the AI assistant you’re currently using, whether that is a default AI assistant built into the interface you’re using or the one you’ve defined yourself.
All tool schemata. All available tools along with their names and descriptions get loaded. This includes everything provided by MCPs (e.g. our Zotero MCP) but also built-in tools that come with the interface (e.g. OpenCode edit tool) or server tools (such as Claude web search).
Skill descriptions. The description of each skill is loaded so that the AI assistant, when needed, can load a skill based on its description. This is called progressive disclosure.
Subagent descriptions. Similarly to the skills, the description of each subagent is included so that the AI assistant can call the subagents based on their descriptions.

All of the above go into the payload that is provided to your chosen model at the beginning of your conversation, and only then is your prompt considered. If you are in an ongoing conversation, all “thinking” and previous turns are also included in this payload. Based on this, you can hopefully see how the context your AI assistant has access to can quickly become full. When we write things like skills, AGENTS.md files, and agent definitions, keeping it simple, avoiding duplication, and splitting off capabilities into subagents where appropriate all aid the AI assistant in assisting you. Filling this stack with too much unrelated information is called , and all of this series comes down to far more than it comes to choosing between which model to use, as it does not matter how powerful the model is if the context is wrong. Or in other terms, a top-tier model like Fable 5 with rotten content will be easily outperformed by a mid-tier model like GLM 5.2 with a carefully curated context.

Next Steps

With all of this in mind, the next steps for you are to write your own AGENTS.md file and an agent definition for the sort of AI assistant you want to be interacting with. Keep in mind to start small. You don’t need a perfect setup to begin with. Start with something simple that works for you and then gradually add more complexity when you need it.

In the next post of this series I will cover some use cases for this system and will trace through examples of how information and knowledge retrieval from the Obsidian vault and from Zotero shapes the AI’s responses and how that aids us in our work.

AI Research Assistant: Runtime (3/5)

Fri, 17 Jul 2026 00:00:00 +0000

In the third part of this series I’m going to cover the specific runtime used to set up our AI assistant. Before going into the details of which parts we need for this I need to state my perspective. My perspective is that we should be in control of the software and that we should reduce dependencies on external parties to minimise the risk of vendor lock-in (becoming dependent on a single provider to the point where switching is costly or impractical). Practically, that means using open-source tools where possible, and routing through providers that respect your data. The following is written from that perspective, but every component I describe is interchangeable and I will mention alternatives where I am aware of those.

Anatomy of an AI Assistant

The first part we need to consider in our AI assistant setup is not the AI model we use, but the difference between model developer and model provider. Of course, with the popular AI models and providers the difference between model developer and model provider is indistinguishable. We shall open that black box. Before I can explain which parts make up that black box, a small note on terminology. The following roles are not mutually exclusive: a company can be developer, provider, and interface provider all at once. But separating them conceptually helps us understand where we have choice and where we do not.

Term	Definition	Example
Developer	Whoever trained the model	Anthropic for Claude, OpenAI for GPT
Provider	Whoever hosts the model and gives access to it	Anthropic’s own API, AWS Bedrock, Microsoft Azure
Router	Routes your request to a provider based on your configuration	OpenRouter, Venice.ai (optional)
Interface	The UI you interact with	ChatGPT, Claude.ai, OpenCode

How a request flows from interface through router to provider and developer.

Anthropic and OpenAI are developers and providers of their respective models while also providing the interfaces used to access them. While this is convenient it does limit flexibility and puts us at the mercy of these companies as they have the power to always change every aspect of it, from model to interface. At the same time, . Fortunately, there are alternatives. Many AI models, whether proprietary (closed-weight) or open-weight, are available through different providers and can also be accessed via a model router. With these terminologies cleared I shall outline each role in more detail and go into some of the options we have.

Developers

At the time of writing this, notable developers of AI models are OpenAI, Anthropic, Google, and Meta. Of these the two most known ones are OpenAI and Anthropic as they developed the leading frontier models GPT and Claude. Often any other models are compared to these two with the question of: But is this model as good as ChatGPT/Claude? In a way they have become what Windows and macOS are to operating systems, the two options everyone knows and most people don’t venture past. This analogy also aligns with a further difference in that ChatGPT and Claude are proprietary models (closed-weight) while other model developers such as DeepSeek also provide their models as open-weights, the free software equivalent for language models. Although mind you, they aren’t open source. Open-weight means that only the trained model weights are publicly available, the training data, training code, and methodology usually stay proprietary. While OpenAI and Anthropic , . DeepSeek deserves special attention here as their models are drastically cheaper than GPT and Claude while being open-weight, widely available across providers, and suitable for our line of work, capable of handling academic research tasks such as literature synthesis and argument checking.

Providers

Providers are the entities hosting AI models on their infrastructure and here we again see a difference in the availability between the models developed as closed-weight models and the open-weight models. For the closed-weight models such as GPT or Claude we can only access these directly through their developers (here they are also acting as providers) or through the infrastructure of hyperscalers (large cloud computing providers) like Amazon or Microsoft that charge Anthropic and OpenAI to host and provide these models to their customers. This does not allow for much competition on pricing. This situation is drastically different when considering open-weight models. Let’s take the example of the model That means anyone can take this model, run it on their own infrastructure, and compete on the pricing under which they make this model available to their clients. Beyond pricing competition, open-weight models under permissive licenses also enable other use cases such as hosting models within a specific jurisdiction with stronger data protection rules (e.g. within Europe), or running them entirely on infrastructure you control where no external party has access to your data, or just ensuring that models cannot be restricted to only certain users by governments, like the and . Additionally, in our line of work, our universities could (and should) be hosting open-weight models on their infrastructure and provide us as academics and their students unlimited free access to these models.

Routers

As there is a huge range of models available by various providers the question becomes which model to choose (I cover that in more detail later) and from which provider. As the field of AI is rapidly changing our choice of model will sometimes change and with that also who provides this model. While just buying a ChatGPT or Claude subscription is convenient, this is limited to the models they provide. Think of a ChatGPT or Claude subscription like having a library card that only lets you read APA journals. They are excellent within that collection, but you can’t access anything else. Instead I argue for the use of a model router, which is like having access to the full library catalogue: you can browse any publisher, any discipline, and pick exactly what you need for each task. A well known example is but there are other options such as , which hosts open-weight models itself and routes requests to closed-weight models. While model routers were originally developed as fallbacks in AI assistant operation for when a model is not currently available to route to another model or the same model by a different provider, a use case that is less relevant for us, they come with other upsides. These are pricing and flexibility. When using a router it can automatically route your request to the cheapest model provider available at the moment and thus save you money and/or only route to providers who don’t retain any of your data. Additionally, it comes with a great deal of flexibility. Imagine a new AI developer publishes a great new model and you want to try it. One way to go about that would be to create an account with that developer and then pay them for access. The alternative is to stick to your router with which you’re already holding an account and credit. One account one bill, regardless of how often you switch between the models provided, who is hosting them, or who developed them.

Interfaces

This brings us to the choice of interface. Well known interfaces are the web or desktop interfaces for ChatGPT and Claude, but there is a wide range of other interfaces for models. We can distinguish between purpose-built interfaces, like the Claude app, and integrations into existing programs, like an IDE (integrated development environment, a code editor) or a note-taking app such as Obsidian. For the purpose of this blog series I will write about purpose-built interfaces, but integrated interfaces can be very useful as well. Interfaces you might have read about are , , , and a recent addition being . These are the proprietary tools from Anthropic and OpenAI has released similar tools with and . The open source community has developed some equivalently useful tools such as (the Claude Code and Codex equivalent), (inspired by Claude Cowork) or (a general-purpose AI agent). The choice of interface is up to you, though it should of course be compatible with the model provider, and possibly router you’ve chosen. Another aspect to keep in mind is the intended purpose of each interface. Interfaces like Claude Code, Codex, and OpenCode are designed for software development and work well for a text-based approach. Tools like Claude Cowork or Open Cowork have built-in capabilities for tasks like generating PowerPoint slides. Think about what you will mostly be doing before choosing. In my case, I use OpenCode (and will base the rest of this series on this choice) which is compatible with OpenRouter (which I’m also using). Furthermore, the interface you choose should also support the Model Context Protocol so that you can connect the tools you’ll need.

Choice of Model

With provider (or maybe router) and interface in place the next choice that needs to be made is which model to use. This is not an easy choice and the choice made will depend on the intended tasks, as such it is more of a choice of multiple models and this section is only intended to enable you to make an informed choice. So then what to base the choice of model on? Two things to consider upfront: modalities and context window.

First, what can the model even process? For most of our work the two main input modalities are text and images. But models vary, some are text-only (like DeepSeek V4 Flash), others handle images, audio, or video (like Gemini 3.5 Flash). If you want your AI to work with graphs or screenshots, you need a multimodal model. If you’re working purely in text, you can save money with a text-only model. Second, the , how much text the model can handle at once. Measured in tokens (the basic units of text that a model processes, roughly equivalent to a word or part of a word), this determines whether a model can process an entire paper, a full codebase, or a long conversation. If your work involves long documents, this will be a deciding factor.

The easy choice is to go with Claude or ChatGPT, or maybe Gemini. A bit more informed choice would be to base this choice on AI benchmark rankings. From these, claims are made such as “this model is best for writing” or “this model is best for programming”. But, all of these benchmarks are based on contrived tasks and do not mean that a model will work for your use case. Instead, I recommend that you develop your own evaluation use case. For instance, someone is using the . Mine is that I give it a few pages of my writing and ask it to verify each and every claim I make against the sources I am citing to support each claim. I then evaluate the AI’s output and judge whether I’m happy with the expected performance.

If you’re looking for a recommendation, I am currently using DeepSeek V4 Flash as a daily driver and DeepSeek V4 Pro or GLM 5.2 if it is a more reasoning intense task, such as checking the argument flow in my writing.

Tools

To borrow a metaphor from the , the model is referred to as the brain and that’s what we have been covering above. In the next steps we’re going to cover the tools, or to stick with the metaphor, the hands. Depending on the choices you made in the above setup some tools might already be available as some interfaces already come with tools. For instance, , but which tools are available can be extended. For providing tools to the AI assistant there are two main ways, the and . There is quite some debate going on over which ones are better, that I won’t get into, (see also ).

Model Context Protocol

The is a standard for integrating external software into an AI assistant. This is the standard we will be relying on to integrate our sources of information into the AI assistant. An MCP can be connected to from an AI assistant either via a local connection or a web connection to an external server. When possible local connections are to be preferred as a remote connection to a server comes with a certain level of risk: the provider can change the tools available through the MCP without you being notified. Imagine your Zotero search suddenly stops working mid-project because the server operator pushed an update. A locally installed MCP avoids this dependency (read more about this in the ). Also, not for everything an MCP is available, that is where agent skills come in.

Agent Skills

are a lightweight, open format for packaging specialised knowledge and workflows. A skill is a folder containing a SKILL.md file, metadata in its frontmatter (name, description), plus step-by-step instructions, along with optional scripts, templates, and reference materials. AI assistants use progressive disclosure which means that at startup, they load only each skill’s name and description and will only load the full instructions when a task matches the skills name and description. This lets AI assistants carry many skills with minimal context overhead, unlike MCPs whose tools are all fully loaded during startup.

Practical Implementations for our Setup

Now, I know that the above is a lot of content to cover but I believe that content needed to be covered to enable you to make an informed choice of how you want to set up your own AI assistant.

In we set up citekeys to match Zotero entries to Obsidian notes. The MCP connections below add a second layer: semantic search across your entire library using , which capture meaning rather than just keywords. Together, citekeys give you precise matching between specific sources and notes, while embeddings let the AI discover sources and notes.

Zotero -> AI

For integrating is available. This MCP works by connecting to Zotero using the local Zotero API. Based on this access to Zotero the MCP software creates a database to index all sources in Zotero, including the full texts of sources if they are stored in Zotero. The text is extracted from those PDFs and turned into embeddings, representations that capture meaning, not just keywords. This means the AI can semantically search your entire library and enables you to ask things like “find papers about the relationship between mind wandering and metacognition” and it will find relevant papers even if none of them use those exact phrases.

Should you happen to use EndNote, , and .

Obsidian -> AI

Connecting Obsidian into our AI assistant works similarly to how Zotero was connected, by MCP. However, to be able to do that we first need to install a plugin for Obsidian, (the developer recently started selling a Pro version that is not needed for our case). This plugin creates embeddings based on the entire content of your vault, similar to how the Zotero embeddings were created. These embeddings can then also be connected into our AI setup using an MCP. That means you can ask the AI things like “find notes about self-regulated learning” and it returns related results even if those exact words never appear. There are based on the Smart Connections plugin. Which exact option to use depends on your use case. The one I am using is . However, they only give the AI assistant the ability to search through your notes, not edit them. For this purpose we will be using, in addition to the MCP, a set of that give the AI the ability to interact with the Obsidian vault. So that in combination your AI assistant will be able to search through your vault but also to make changes to it.

Should you use other note taking tools there also are options available, such as an , an , and .

Search -> AI

Lastly, in addition to the connections to Obsidian and Zotero we need one more MCP, for searching the internet. When thinking about searching the internet the first thing that usually comes to people’s mind is Google. However, . There are alternative search providers, some specific for usage by AI, such as or . While using Exa, Firecrawl or alternatives is an option, I personally prefer, and recommend, . , for both you and your AI assistant. Kagi, or other search providers, can also be added to our AI assistant using the , , or a similar integration for the search provider you prefer.

Next Steps

The next step is for you to choose which route you’re going to take. The point of this series is to enable you to set up your own AI assistant and that starts by making choices about the content covered above. How do you want to access the models you’re using? Do you want to use a router? Do you have requirements about the provider (e.g. location or data storage)? Which interface do you want to use? I recommend you start with what you’re already using and evaluate if it is still suitable for your purpose and workflow. That will give you a starting point.

If you’re looking for a recommendation, I am using OpenRouter to access models only from providers with a zero-retention policy (meaning the provider does not store your inputs or outputs after the request is completed). These I access through OpenCode and nowadays I mostly use DeepSeek V4 Flash.

With choices made about how to access models, which model to choose in which interface, and how to connect Obsidian, Zotero, and Kagi, the infrastructure is complete. What remains is the AI assistant’s, for lack of a less anthropomorphic word, personality: its role, its rules, its voice. That is the subject of the next post.

AI Research Assistant: An Overview (1/5)

Fri, 10 Jul 2026 00:00:00 +0000

Over the last few months (or, in a way, years) I’ve been working on my local setup for using AI to assist me in my research. I have shown this setup to a few colleagues and they have asked me to write about it and provide workshops on how to recreate such an AI research assistant for themselves. As such, I am writing this series of blog posts to structure my thinking, document my setup, but also to point future workshop participants to it as a resource. This series is written for researchers who are open to using AI to assist them in their work. While a similar setup could be used for other purposes, such as teaching, this is not what it was designed for. Some basic IT familiarity is required, such as installing software from GitHub, but everything can be put together by following the instructions provided with each piece of software. This post serves as an overview of my setup and the blog post series. This project has been inspired by other projects, notably the and the . Neither project fit my workflow and both are, in my opinion, too complex as they are aiming to achieve too much in one project. This led me to building my own setup.

There are two main aspects to this work, Personal Knowledge Management (PKM) and AI, and I maintain that AI only becomes really powerful when based on PKM. Without the required knowledge about your work any AI system is only as good as its training data. Or in other words, any AI is only ever as good as the information it has available. Garbage in, garbage out. This is also called (the practice of curating what information an AI has access to so that its output is grounded in your work rather than its training data alone).

Architecture overview of the AI research assistant setup.

Sources of Information

The second blog post in this series outlines how to set up a basic personal knowledge management system for the purpose of informing any local AI setup while also being helpful to you. When considering what knowledge will be useful when using an AI in the context of research a few things come to mind. Possible candidates are literature databases and scholarly papers. However, these are not who you are and do not carry your point of view. Therefore, the most important source of knowledge are your notes. The crucial aspects of this setup will be that it is machine readable and structured. Example content that I have in my PKM are things like all my publications (including any major versions and reviews), grant applications, reviews I’ve written, notes I’ve taken about literature I’ve read, and notes I took during meetings or conferences. These capture my point of view and enable any AI ingesting these texts to align its answers with my perspective on the content. A beneficial side effect (and in my case the original purpose) of setting up such a PKM is that it not only makes it possible to serve as context for an AI assistant, it also makes it easier for you to retrieve your previous work and notes.

Runtime

With a knowledge base in place we can turn to the third blog post in this series, the specific runtime, or in other words, the software we’ll be using. In this post we will first break down the different roles involved in an AI assistant (developers, providers, routers, and interfaces), then cover how to choose between them and which model to use, and finally how to connect the combination to your knowledge base and other relevant tools we might already be using, such as Zotero, and to web search. This is very much the “how” of putting all the pieces together, but mind you, without all the pieces (such as the PKM) in place this setup will be far less useful. I will also further outline that every piece of this setup is interchangeable, we should not be locked into any ecosystem or provider and as such this setup is designed so that every part can be exchanged.

Agent Definition

Once a PKM, or any other machine readable source of knowledge you maintain about your work, is available alongside an implementation for the runtime, the next step is to work on the instructions for any future AI and this will be the topic of the fourth blog post in this series. We will cover the role of an AGENTS.md file and how this is different from an agent definition. This enables us to define in your local setup which role you want your AI assistant(s) to fill. This agent definition can be supplemented by subagents specific to your use case. The important part here is that you define what you need based on your workflow and to ensure that it is your use case that drives this process. I will also cover how all of these pieces come together in each session when you prompt the AI assistant.

Use Cases

Lastly, in the fifth blog post, I will cover some use cases that have been useful to me and trace how pieces of information flowed from knowledge base or tool to inform the AI generated answer. Further I will cover some temptations worth resisting when it comes to responsible and ethical use of AI in academic work. Finally, I will outline some directions I am considering for future development.

Takeaway

The take away for you as the reader I am aiming for is not a detailed guide of how you can set up the exact same system I am using, the goal is to get you to think through which parts you need and fit into your workflow. Maybe you use a different reference manager or your institution provides you with a local AI system (lucky you). Either way, after reading this series you should be able to make an informed decision of how to structure your knowledge base so that it works for you and an AI and to identify which integrations to set up to connect your knowledge sources to your preferred model.

AI Research Assistant: Sources of Information (2/5)

Fri, 10 Jul 2026 00:00:00 +0000

In the second part of this series I will cover which sources of information can be fed into an AI assistant setup. In this part I cover how to set these up and connect them with each other, and in the next post how to connect them to your AI assistant.

In a way this post can stand alone, covering only sources of information and how to manage them (whether or not you’re using AI) and in the next post I will cover the AI-specific parts. While I will point out alternatives to the software I use, if I am aware of alternatives, I will focus on how these pieces fit together with the software I use as I can’t cover all possible variations. In addition to what I describe a simple text editor will also be useful (e.g. I use but something like would do). But, at first we need to establish what sources of information are relevant.

As academics our primary source of information is the academic literature. We always have new papers to read and a reading list so long we will never cover it. Not only that, each and every one of us has probably read more papers than they remember and can actively draw on themselves. That is very human. But, even if we cannot directly recall every part of what we’ve read, what we’ve read (and of course what we’ve learned) shapes our thinking. However, we cannot hope that feeding those same papers to an AI results in it generating output that is aligned with our thinking. If we wanted to achieve that we need a more personal form of knowledge and what I mean by that are your notes. Many of us take notes when reading a paper, listening to a talk, or when going about our daily work. These notes contain our points of view, our appraisal of someone’s work, what you’re currently working on and more. An AI that can draw upon these will thus be able to generate answers based on those. Thus, for an AI to be useful to us it needs access to both the information in the literature we built our work on and the personal knowledge in our notes. And both ways of structuring information I outline below will not only enable you to easily feed them into an AI, they will also make it easier for you to find relevant information without AI. That prompts the question: how do we provide access to this information to an AI? The answer to this consists of three parts: information management (reference management), personal knowledge management, and how to connect them.

Information Management (Reference Management)

The first part of that is access to the literature we’ve built our work on. This is not as easy as pointing an AI at an academic database because if we give the AI access to an academic database then this database is no longer a curated source of information and it is this curation that adds value. We can of course upload a bunch of PDFs but that risks overloading the AI’s (the amount of text a model can process in a single request). A practical answer to this conundrum is to use a reference manager such as (my recommendation), , or . These reference managers allow us to structure our collection of sources and to curate which sources are included. Thus it is useful to both us and our AIs. When using the reference managers it is important that these are not only used for references based on metadata, but that they contain the full text PDFs of each source. These references then need to be meaningfully sorted, in this case, in a way that is meaningful to you. For example, in my case I have folders for self-regulated learning and mind wandering and within each of those multiple subfolders depending on source category. An AI will later be able to query both on storage location and metadata. Thus, the more sorted your work is and the better quality the metadata provided, the easier an AI will be able to find the relevant papers.

Personal Knowledge Management

Once we have organised our literature, the next step is to organise our notes in what is called Personal Knowledge Management (PKM). PKM refers to a system of capturing, organising, retrieving, and synthesising knowledge relevant to you. Capturing could be writing a note about a paper, organising would be to provide it metadata and tags that fit into your organisational system (again, something that makes sense to you). Retrieving is, as the name suggests, being able to find that knowledge again, usually based on how you organised it. Lastly, synthesis refers to compiling the organised information into new work, in our case usually writing or new ideas.

There are , a famous one you might have heard about is the . I will not go into the details of various methods as the details for this are out of scope for this series. I will, however, cover the relevant parts for how a PKM can be integrated into an AI setup. These are machine readability, which software to use, version control, metadata, and file organisation.

Machine Readability

The first part of a PKM to cover is machine readability. Machine readability refers, as the name suggests, to formats that are easily readable by a machine, as AIs are machines. PDFs do not fall in this category and when you upload a PDF to an AI this gets converted to machine-readable content in the background. The point of machine readability is closely linked to the software you use for your notes and your future (or existing) PKM. Popular note-taking software includes things like , , or just Word files in a folder. These are not examples of software that stores its content in a machine-readable format. There also is a variety of software available for PKM with popular examples being (what I use), , and . A decisive point of difference is how this software stores information. Something like Notion is cloud software where the content is not easily accessible for an external machine while Obsidian and Logseq store their data in plain text Markdown files. is a plain text format used to indicate in plain text files how text is structured in terms of headings; additionally, formatting such as bold or italics can be specified. There are various flavours of Markdown, such as or that can be used for our purpose. Markdown is in essence nothing but plain text and an AI can always easily read Markdown text, in fact, it is a common format that AIs use in their answers. That means that a PKM that stores its knowledge in Markdown files (.md file extension) is not only easily readable by a human but also easily readable by an AI, perfect for our aim of a local setup that feeds our notes into an AI.

Software

Having the requirement in place that the notes in our PKM need to be machine-readable files severely limits our choice of PKM or note-taking software. Popular options such as OneNote or Notion store our notes in their cloud services or proprietary formats. Nevertheless, we still have some options when it comes to a local-first PKM software that supports Markdown. Options that I am aware of are , , , and . I am using Obsidian and chose it for its huge and active community, but you should be able to do most, if not all, of what I’m going to outline in this series with either of those options. Another advantage of Obsidian, tied to its active community, is that there are lots of plugins available that you can use to customise Obsidian to your liking. Should you be interested in those, a , although some of those might be a bit much if you’re just getting started.

Before moving on, a brief tip. If you’re planning on converting Word files into markdown for inclusion in your vault and those Word files contains citations managed by a reference manager, then have a look at the on how to preserve those citations in markdown.

Version Control

Following from picking a software for PKM, the next step is version control. You might be used to adjusting file names with a suffix in a pattern like filename_v4.docx or something along those lines and that is not what I mean here. What I mean is specialised version control software such as . This sort of software was originally developed for software development, but given that our notes are plain text, the same sort of software will also work well for our purposes. A full is out of scope for this post but the main points that are relevant for our context are that it allows you to basically take a snapshot of the current state of your notes that is time-stamped and then push that current state to a remote storage location. A popular remote for version control is but if you choose that make sure to configure your repository to be private, otherwise your private notes will be visible to everyone. Having our notes version controlled comes with various advantages but the main one is that you can go back in time to previous versions should you need to, which is especially useful when the AI you use produces errors, and that is inevitable, all AIs produce errors at some point, and then having access to the previous versions allows you to easily undo all changes the AI made. Imagine working on a grant application and the AI adds a factual error or generates irrelevant output, with version control in place you can easily and quickly revert. Should you want to use version control with Git from within Obsidian, there is a available for that purpose. Lastly, if you choose to use a different remote than GitHub, make sure you have enough space. To give you a frame of reference, I’ve been doing this for 4 years now and my notes folder is about 1GB in size.

Metadata

Now that we have a format that an AI can read and software to manage these notes, the next question is what it reads. The main part of these notes will of course be the content resulting from your work, but it needs to be able to find this content first. This is where metadata comes into play. Metadata is data about data, in this case data about your notes. This could look like (following the ), where metadata is stored at the top of a file in a format called YAML (a simple, human-readable key-value format):

type: journal-article
source: Metacognition and Learning
authors: Daniel Ebbert, Negin Mirriahi, Natasha Wilson, Malgorzata Korolkiewicz, Nicholas Fewster-Young, Anna Lloyd, Maurizio Costabile, Srecko Joksimovic, Shane Dawson
year: 2026
title: Mind Wandering in Video-Based Learning: Self-Regulated Learning and Student Responses in a Naturalistic Setting
citekey: @ebbertMindWanderingVideoBased2026
tags: #self-regulation #mind-wandering

These metadata will allow an AI, and you (see the retrieval aspect of PKM) to find the relevant information. Want to get the notes related to the person Daniel Ebbert? Query by the name. Want to find all sources related to mind wandering? Query by the tags. What exact metadata you enter will depend on your use case and the information you want in your PKM, but if your notes are about literature you’ve read I recommend to at least cover the title of the source, who wrote it, when it was published and where, along with some tags so that you can find it more easily later.

Organisation

Next, similar to how you sorted your literature into folders, your notes also need to be sorted. For example, I have one folder for my daily notes, another for my grant applications, another for my publications, another for each talk I’ve given, another for my literature notes, another for the reviews I’ve written, and so on, you get the picture. The specific folders used here are less important than that they make sense to you and serve a specific purpose. Clearly distinct folder organisation is beneficial here as then an AI, and you, can search for specific information within specific folders. For example, me asking my AI to summarise the main reasons why my submissions were rejected can then be answered by the AI querying for the type: peer-review metadata in combination with filtering on the folder that contains the full texts and reviews of my publications.

Information Matching

Once you have both your reference manager and your PKM set up the next step is to connect them so that the sources in your reference manager can be matched to your notes in the PKM. Both directions of connection can be established and in the following I will showcase how to accomplish this using Zotero and Obsidian.

Citekeys

The challenge when matching information between systems is that a common identifier is needed, in this case for sources we have read and written notes about. For this purpose we will be using citekeys from . BibTeX refers to a file format (.bib) for bibliographic information, such as journal articles or conference proceedings. In this file format each entry is assigned a unique identifier called a citekey. Then, when writing (originally this was designed for ) an author can refer to a piece of work by its citekey, for example @ebbertMindWanderingVideoBased2026 refers to the paper in the metadata example above. Although the citekey construction is arbitrary, it is useful if it contains the name of the first author and the year, but don’t worry, this can be automated. For the purpose of connecting Zotero and Obsidian we’re going to start with a Zotero plugin called . Once you have this plugin installed in Zotero it will automatically generate citekeys for everything you have stored in Zotero. Furthermore, it allows you to export your entire library as a .bib file, making it machine-readable so that it can be picked up in Obsidian.

Citekeys in Zotero and Obsidian enable matching sources to notes.

Zotero -> Obsidian

With the Better BibTeX plugin in place we can move on to setting up literature notes in Obsidian based on sources we have stored in Zotero. This can be accomplished using an . This plugin allows for the creation of literature notes in Obsidian based on metadata, notes, and PDF annotations from Zotero. It creates such a note based on a template for this purpose. I recommend you customise this template to your liking based on the and . If you’re curious, .

Two crucial things I would like to point out in this template:

The line Zotero PDF Link: {{pdfZoteroLink}} generates a link in Obsidian that conveniently opens the corresponding PDF in Zotero.
The metadata for each note created using this template contains this line Citekey: "@{{citekey}}". That means that each literature note will contain the citekey and thus both you and AI can match sources and notes between Zotero and Obsidian based on this citekey.

Obsidian -> Zotero

Once we have created a few literature notes in Obsidian we can set up the connection back to Zotero using another Zotero plugin called . This plugin scans a folder with Markdown files for files that contain a specified identifier, by default the citekey. Then, for each source in Zotero with a corresponding note, it adds a custom tag. I called this tag noted. and I chose purple for the tag noted. What this means in practical terms is that I can look at the list of literature in Zotero and can quickly see for which sources I’ve already created notes.

Colour-coded Zotero tags (purple = noted, blue = skimmed, green = read) provide an at-a-glance reading status overview.

In my case I’ve also added the coloured tags of blue for skimmed and green for read allowing me to track my reading status. Additionally, the MarkDB-Connect plugin adds the ability to directly open a note from Zotero. So if I see that I’ve made a note about a source, all I need to do to open this note is to right click on the source and in the context menu select the option Open Markdown Note.

With this piece in place we have now set up both Zotero and Obsidian and connected them to each other and enabled that sources can be matched with their notes by both you and an AI.

Next Steps

With all of this set up I hope that you can see how such a setup might not only inform an AI setup but will also be beneficial to you regardless of whether you’re using AI or not. The next step is for you to determine where you are at in the process of structuring your knowledge base, maybe you already use a reference manager or maybe you just have a folder with a lot of PDFs, or maybe you already have detailed notes about your work but it isn’t in a machine-readable format. The starting point will be different for each and every one of you, but based on the above you should be able to figure out where to start on your personal setup. In the next post I’ll cover the runtime, including how to choose between models, providers, and interfaces, and how to connect this local information management system to your personal AI setup, but, before that, I want to acknowledge that information and knowledge management today sits on a spectrum between fully human-controlled and AI-managed.

The above is written from the perspective of full human control over knowledge management. That was the case when I set up the system I describe and is reflected in how I structured my work and which metadata I use. The fact that it works well for feeding this knowledge into an AI was not the original intent; it is merely a beneficial side effect. Nowadays, I still maintain most of the control and write the content in my notes largely manually, but sometimes I instruct my AI assistant to fill a note based on a conversation or to make sure I have cross-references in place. However, I recognise that there are different takes on this, and you might want to position yourself differently on this spectrum. If you wish to give the AI assistant more control over the knowledge management, then I encourage you to read about and the . As part of this conversation, the recently introduced is also interesting. While it is inspired by Andrej Karpathy’s LLM wiki, it is framed as humans staying in charge of creating OKF bundles that are then fed into an AI. It will be interesting to see how this format develops.