Karim Eldefrawy

Cryptography, Cybersecurity, Privacy Computer Science

Co-founder & CTO of Confidencial.io
2017-2021: SRI (Internet Origin)
2011-2016: HRL (Part of IBM Research)
2006-2010: PhD@UC Irvine

Scientific curiosity

Article 11 of 17 · Draft

Papers, Patents, Prototypes—and Vanishing Capability

The artifacts that remain easiest to count after a team disperses are not the same as the capability needed to produce, challenge, and transition the next result.

Countable research outputs cast a partial shadow while teams, tools, memory, correction, and transition paths remain behind the measurement plane.
Conceptual illustration; output counts are evidence categories, not a performance score.

One output, several domains

Thirty-one granted patents distribute across five disclosed technical topics

The distribution demonstrates breadth in a countable output class. It does not measure patent quality, independence, adoption, revenue, public value, or the current existence of the teams that produced them.

Primary-topic classification on the author-maintained site; several patents could reasonably span more than one category. Inspect the source record.

Source-linked argument map

Why an output ledger cannot serve as a capability balance sheet

Outputs are real evidence, but every metric omits critical state and can be optimized without preserving the production system.

  1. Documented observation

    The site contains many countable artifacts

    The author’s records list 79 research works, 31 granted patents, and 12 funded R&D awards.

    Karim Eldefrawy Karim Eldefrawy Karim Eldefrawy

  2. Measurement distinction

    Each artifact answers a different question

    A paper records a claim, a patent a legally examined invention, a program funded activity, and an award retrospective recognition.

    U.S. Patent and Tra... Network and Distrib...

  3. Proposed mechanism

    Proxy selection reshapes behavior

    When renewal and status depend on visible outputs, institutions can increase the proxy while underinvesting in maintenance, correction, apprenticeship, and transition.

    Diana Hicks, Paul W... Jian Wang, Reinhild...

  4. Bounded conclusion

    Pair every output with a state transition

    Evaluation should ask what became true, usable, maintained, teachable, and reproducible—and which capability remained five years later.

    Network and Distrib... Diana Hicks, Paul W...

The framework does not diminish the work represented by outputs; it limits what may be inferred from their counts.

My website lists 79 research works, 31 granted U.S. patents, and 12 funded R&D awards. It also records software, talks, awards, and a company built from earlier research. These are not invented metrics imposed from outside; I chose to curate them because they record real work. D

This disclosed record is an audit object, not a representative benchmark. It is useful because the claims and omissions can be inspected against a known ledger. Nothing in the counts establishes what another researcher, field, institution, or career stage should produce.

They also demonstrate the measurement problem.

The lists remain after a project ends. They remain after colleagues move. They remain if code no longer builds, an instrument is dismantled, a result is superseded, or no organization can reproduce the integrated system. That durability makes outputs useful historical evidence and dangerous substitutes for capability.

What each count actually certifies

The publication record establishes that the site curates 79 works. It does not establish that every result is correct, equally important, independently replicated, or still supported by an active research line.

The patent record links 31 granted inventions to public patent documents. A granted patent has passed a legal examination for patentability under the relevant process. The USPTO’s guidance makes clear that this is patent examination, not scholarly peer review. A patent does not certify that the invention was built, adopted, profitable, secure under every model, or institutionally preserved.

The project record lists 12 funded awards. An award establishes that a sponsor selected and funded work under stated conditions. It does not establish that the program met every goal, transitioned, or left reusable capability.

These are not criticisms of the outputs. They are type checks. A system becomes confused when evidence of one kind is accepted as a verdict of another.

Outputs are projections

Imagine the actual research system as a high-dimensional state:

  • people and complementary roles;
  • working relationships and mentorship;
  • code, data, instruments, fabrication, and test environments;
  • tacit knowledge and negative results;
  • authority to choose problems;
  • access to users and transition partners;
  • independent mechanisms for finding error.

A publication count projects that state onto one axis. A patent count projects it onto another. Funding totals, prototypes, citations, and awards are additional projections. None is false. Each discards information.

The danger appears when the projection becomes the objective. Teams learn which work produces countable units. Institutions allocate support to what renews grants, improves rankings, fills an IP ledger, or produces a demonstration before review. Maintenance and replication remain undercounted because they often preserve value rather than create a new unit.

The Leiden Manifesto does not reject quantitative indicators. It argues that indicators should support qualitative expert assessment, respect field differences, and remain open to scrutiny. Research on delayed recognition and novelty adds another warning: novel work can have more variable impact and take longer to be recognized, so short windows systematically distort selection. D

The capability-lag problem

Output and capability can move in opposite directions for a time because output lags accumulated state.

A mature team may publish and patent vigorously while losing junior hiring, maintainers, or discretionary time. Previously initiated projects continue yielding results. The visible ledger looks healthy. The missing investment becomes apparent only when the next generation of problems fails to start.

This is analogous to a factory meeting shipments while deferring maintenance. The analogy has limits—research is not a production line—but the accounting insight holds. Current output draws on past capability. It does not measure the replacement rate.

A minimal institutional report should therefore pair flows with stocks:

  • Flow: papers, patents, prototypes, dollars, milestones, hires, releases.
  • Stock: intact teams, maintained artifacts, functioning facilities, mentorship depth, active transition relationships, and time-to-reconstitute lost capability.
  • Depreciation: departures, obsolete environments, inaccessible data, broken interfaces, lost problem authority, and unrecorded failures.

Without the last two, rising output may conceal capability debt.

A retrospective signal and its boundary

The NDSS Test of Time Award is a rarer kind of evidence. It recognizes influence after a long interval rather than optimizing for immediate attention. In 2024 it recognized work I coauthored on an Internet-censorship system. That is meaningful evidence of enduring scholarly and technical influence. D

It still does not answer every question. Did the system deploy? Which ideas were reused? Did the original team remain connected? Did the institution preserve the capacity to pursue the next problem? What public benefit followed? A retrospective award strengthens one verdict without collapsing the other four.

That discipline protects the work from both inflation and erasure. We need not pretend an award proves everything in order to say that it proves something important.

The strongest counterargument

Research capability is latent. Unlike a paper count, it cannot be audited directly. Managers who dislike an objective metric can invoke “tacit knowledge” or “future potential” to protect favored teams. A capability ledger could become a vocabulary for institutional rent-seeking.

That is a real danger. The answer is not to retreat to one-dimensional counts. It is to operationalize the stock.

For each claimed capability, ask for observable tests:

  • Can an independent team build or reproduce the artifact?
  • Which later projects reused it?
  • How many critical roles have credible successors?
  • What is the time and cost to restore the environment?
  • Can the group initiate a new problem rather than only service inherited tasks?
  • Has a user, sponsor, or product team accepted responsibility for transition?
  • Which claim was corrected because the institution’s error-detection process worked?

These questions do not eliminate judgment. They make judgment contestable.

Five different verdicts

  • Scientific success: Publications and retrospective recognition provide evidence of new and influential knowledge, with correctness and importance remaining claim-specific.
  • Technical success: Patents, code, and prototypes provide evidence of disclosed or working mechanisms, not universal system performance.
  • Transition success: A spinout or deployment requires separate evidence of adoption, operation, and maintenance.
  • Institutional success: Output records say little by themselves about whether teams, tools, and apprenticeship survived.
  • Public-value success: Citations, standards, safer systems, commercial use, and trained people may each carry public value that the originating organization captures only partly.

What the successor must learn

The successor laboratory should make every major review two-sided. The output ledger asks what was produced. The capability balance sheet asks what the institution can now do that it could not do before—and what it can no longer do despite the outputs it retains.

No composite score should hide the answer. A high patent count cannot compensate for an irreproducible system. A maintained team cannot compensate for years without a hard external result. The dimensions should remain visible so governance must confront the tradeoff.

The uncomfortable open question is personal and institutional: which item on my own output lists represents a living capability today, and which is now only a durable record that the capability once existed?

Argument under pressure

Two rounds of objection—not a ceremonial counterargument

These are simulated steelman exchanges. The second objection responds to the first answer; the conclusion is narrowed where the objection survives.

Claim under test

Output metrics can rise while research capability declines.

  1. First-level objection

    Papers, patents, programs, and awards are not arbitrary proxies; they are costly artifacts that correlate with real work.

  2. Response

    Yes. They should remain evidence. The claim is that no one output class identifies the retained system or its future production capacity.

  3. Second-level objection

    Adding qualitative capability judgments invites favoritism and makes evaluation less auditable than counts.

  4. Bounded conclusion

    Use structured, inspectable evidence—artifact reuse, team continuity, maintenance, independent correction, transition, and rebuild cost—alongside counts, with reviewers required to explain deviations.

Diana Hicks, Paul Wou... U.S. Patent and Trade...

Claim under test

A Test of Time Award is stronger evidence than near-term citations.

  1. First-level objection

    Awards remain selective social judgments and can reflect network visibility, field size, or survivorship bias.

  2. Response

    Correct. Long delay filters some novelty cycles and offers evidence of influence, but it does not prove correctness, deployment, or institutional continuity.

  3. Second-level objection

    If even retrospective recognition cannot support a durable verdict, the framework risks making success impossible to establish.

  4. Bounded conclusion

    Allow strong but dimension-specific conclusions: the award supports enduring scholarly influence; separate evidence must establish technical use, transition, institutional survival, and public value.

Network and Distribut... Jian Wang, Reinhilde ...

Revision ledger

Corrections and feedback incorporated

Inspectable claims

Source Ledger

  1. Peer-reviewed empirical study

    Bias against Novelty in Science: A Cautionary Tale for Users of Bibliometric Indicators

    Jian Wang, Reinhilde Veugelers, and Paula Stephan. Supports: Evidence that novel research has more variable impact, delayed recognition, broader cross-field influence, and disadvantages under short-window bibliometric evaluation.

  2. Research-evaluation principles

    Bibliometrics: The Leiden Manifesto for Research Metrics

    Diana Hicks, Paul Wouters, Ludo Waltman, Sarah de Rijcke, and Ismael Rafols. Supports: The principle that quantitative indicators should support, not replace, qualitative expert assessment and should reflect field differences.

  3. Agency primary source

    Applying for Patents

    U.S. Patent and Trademark Office. Supports: The statutory examination of patent applications for patentability, which is distinct from scholarly peer review of research publications.

  4. Author-maintained research record

    Publications by Research Area

    Karim Eldefrawy. Supports: The site's catalog of 79 research works; the record establishes countable outputs but does not by itself establish adoption, correctness, or retained institutional capability.

  5. Author-maintained invention record linked to USPTO records

    Patents

    Karim Eldefrawy. Supports: The site's list of 31 granted U.S. patents and their linked patent records; a grant establishes patentability, not scientific peer review, product adoption, or institutional continuity.

  6. Author-maintained project record

    Extramurally Funded Projects

    Karim Eldefrawy. Supports: The site's record of 12 funded R&D awards across HRL, SRI, and Confidencial.io; the list records funded activity rather than long-term transition or capability survival.

  7. Awarding-organization record

    NDSS Test of Time Award

    Network and Distributed System Security Symposium. Supports: The award's purpose of recognizing influential work from prior NDSS symposia and the 2024 recognition of the Internet-censorship system paper coauthored by the author.

Public record

Version history

Published versions are immutable snapshots. Corrections create a new version rather than silently changing the historical record.

No public snapshot has been archived yet. The first snapshot is created when the article is published.

Read the correction, withdrawal, and versioning policy

← Return to the series map

This essay distinguishes firsthand memory, documentary evidence, interviews, and analysis. Published versions remain available even after revision or withdrawal.