Merge branch 'main' into fix/return-processing-no-response

Author: swcurran

.github/workflows/pip-audit.yml

@@ -16,8 +16,9 @@ jobs:
         run: |
           python -m venv env/
           source env/bin/activate
+          python -m pip install --upgrade pip
           python -m pip install .
-      - uses: trailofbits/gh-action-pip-audit@v0.0.4
+      - uses: pypa/gh-action-pip-audit@v1.0.0
         with:
           virtual-environment: env/
           local: true

AnoncredsProofValidation.md

@@ -0,0 +1,84 @@
+# Anoncreds Proof Validation in Aca-Py
+
+Aca-Py does some pre-validation when verifying Anoncreds presentations (proofs), some scenarios are rejected (things that are indicative of tampering, for example) and some attributes are removed before running the anoncreds validation (for example removing superfluous non-revocation timestamps).  Any Aca-Py validations or presentation modifications are indicated by the "verify_msgs" attribute in the final presentation exchange object
+
+The list of possible verification messages is [here](https://github.com/hyperledger/aries-cloudagent-python/blob/main/aries_cloudagent/indy/verifier.py#L24), and consists of:
+
+```
+class PresVerifyMsg(str, Enum):
+    """Credential verification codes."""
+
+    RMV_REFERENT_NON_REVOC_INTERVAL = "RMV_RFNT_NRI"
+    RMV_GLOBAL_NON_REVOC_INTERVAL = "RMV_GLB_NRI"
+    TSTMP_OUT_NON_REVOC_INTRVAL = "TS_OUT_NRI"
+    CT_UNREVEALED_ATTRIBUTES = "UNRVL_ATTR"
+    PRES_VALUE_ERROR = "VALUE_ERROR"
+    PRES_VERIFY_ERROR = "VERIFY_ERROR"
+```
+
+If there is additional information, it will be included like this:  `TS_OUT_NRI::19_uuid` (which means the attribute identified by `19_uuid` contained a timestamp outside of the non-revocation interval (which is just a warning)).
+
+A presentation verification may include multiple messages, for example:
+
+```
+    ...
+    "verified": "true",
+    "verified_msgs": [
+        "TS_OUT_NRI::18_uuid",
+        "TS_OUT_NRI::18_id_GE_uuid",
+        "TS_OUT_NRI::18_busid_GE_uuid"
+    ],
+    ...
+```
+
+... or it may include a single message, for example:
+
+```
+    ...
+    "verified": "false",
+    "verified_msgs": [
+        "VALUE_ERROR::Encoded representation mismatch for 'Preferred Name'"
+    ],
+    ...
+```
+
+... or the `verified_msgs` may be null or an empty array.
+
+## Presentation Modifications and Warnings
+
+The following modifications/warnings may be done by Aca-Py which shouldn't affect the verification of the received proof):
+
+- "RMV_RFNT_NRI":  Referent contains a non-revocation interval for a non-revocable credential (timestamp is removed)
+- "RMV_GLB_NRI":  Presentation contains a global interval for a non-revocable credential (timestamp is removed)
+- "TS_OUT_NRI":  Presentation contains a non-revocation timestamp outside of the requested non-revocation interval (warning)
+- "UNRVL_ATTR":  Presentation contains attributes with unrevealed values (warning)
+
+## Presentation Pre-validation Errors
+
+The following pre-verification checks are done, which will fail the proof (before calling anoncreds) and will result in the following message:
+
+```
+VALUE_ERROR::<description of the failed validation>
+```
+
+These validations are all done within the [Indy verifier class](https://github.com/hyperledger/aries-cloudagent-python/blob/main/aries_cloudagent/indy/verifier.py) - to see the detailed validation just look for anywhere a `raise ValueError(...)` appears in the code.
+
+A summary of the possible errors is:
+
+- information missing in presentation exchange record
+- timestamp provided for irrevocable credential
+- referenced revocation registry not found on ledger
+- timestamp outside of reasonable range (future date or pre-dates revocation registry)
+- mis-match between provided and requested timestamps for non-revocation
+- mis-match between requested and provided attributes or predicates
+- self-attested attribute is provided for a requested attribute with restrictions
+- encoded value doesn't match raw value
+
+## Anoncreds Verification Exceptions
+
+Typically when you call the anoncreds `verifier_verify_proof()` method, it will return a `True` or `False` based on whether the presentation cryptographically verifies.  However in the case where anoncreds throws an exception, the exception text will be included in a verification message as follows:
+
+```
+VERIFY_ERROR::<the exception text>
+```
+

CHANGELOG.md

@@ -1,7 +1,66 @@
+# 1.0.0-rc0
+
+## August 18, 2022
+
+1.0.0 is a breaking update to ACA-Py whose version is intended to indicate the
+maturity of the implementation. The final 1.0.0 release will be Aries Interop
+Profile 2.0-complete, and based on Python 3.7 or higher. The initial (rc0)
+release candidate is for early adopters to provide feedback.
+
+### Breaking Changes
+
+As of rc0, there are no breaking updates in the release from the previous v0.7.4.
+However, we know that there are some pending updates that will be breaking, hence
+the bumps to the major/minor version elements.
+
+### Categorized List of Pull Requests
+
+In rc0, there are not a lot of new features, as the focus is on cleanup and
+optimization. The biggest is the inclusion with ACA-Py of a universal resolver
+interface, allowing an instance to have both local resolvers for some DID
+Methods and a call out to an external universal resolver for other DID Methods.
+While some work has been done on moving the default Python version beyond 3.6,
+more work is still to be done on that before the final v1.0.0 release.
+
+### Categorized List of Pull Requests
+
+- Verifiable credential and revocation handling updates
+  - Refactor ledger correction code and insert into revocation error handling [\#1892](https://github.com/hyperledger/aries-cloudagent-python/pull/1892) ([ianco](https://github.com/ianco))
+  - Indy ledger fixes and cleanups [\#1870](https://github.com/hyperledger/aries-cloudagent-python/pull/1870) ([andrewwhitehead](https://github.com/andrewwhitehead))
+  - Refactoring of revocation registry creation [\#1813](https://github.com/hyperledger/aries-cloudagent-python/pull/1813) ([andrewwhitehead](https://github.com/andrewwhitehead))
+
+- DID Registration and Resolution related updates
+  - feat: add universal resolver [\#1866](https://github.com/hyperledger/aries-cloudagent-python/pull/1866) ([dbluhm](https://github.com/dbluhm))
+  - fix: resolve dids following new endpoint rules [\#1863](https://github.com/hyperledger/aries-cloudagent-python/pull/1863) ([dbluhm](https://github.com/dbluhm))
+  - fix: didx request cannot be accepted [\#1881](https://github.com/hyperledger/aries-cloudagent-python/pull/1881) ([rmnre](https://github.com/rmnre))
+
+- Internal Aries framework data handling updates
+  - fix: update RouteManager methods use to pass profile as parameter [\#1902](https://github.com/hyperledger/aries-cloudagent-python/pull/1902) ([chumbert](https://github.com/chumbert))
+  - Allow fully qualified class names for profile managers [\#1880](https://github.com/hyperledger/aries-cloudagent-python/pull/1880) ([chumbert](https://github.com/chumbert))
+  - fix: unable to use askar with in memory db [\#1878](https://github.com/hyperledger/aries-cloudagent-python/pull/1878) ([dbluhm](https://github.com/dbluhm))
+  - Enable manually triggering keylist updates during connection [\#1851](https://github.com/hyperledger/aries-cloudagent-python/pull/1851) ([dbluhm](https://github.com/dbluhm))
+  - feat: make base wallet route access configurable [\#1836](https://github.com/hyperledger/aries-cloudagent-python/pull/1836) ([dbluhm](https://github.com/dbluhm))
+  - feat: event and webhook on keylist update stored [\#1769](https://github.com/hyperledger/aries-cloudagent-python/pull/1769) ([dbluhm](https://github.com/dbluhm))
+
+- Unit, Integration and Aries Agent Test Harness Test updates 
+  - Fixes a few AATH failures [\#1897](https://github.com/hyperledger/aries-cloudagent-python/pull/1897) ([ianco](https://github.com/ianco))
+  - fix: warnings in tests from IndySdkProfile [\#1865](https://github.com/hyperledger/aries-cloudagent-python/pull/1865) ([dbluhm](https://github.com/dbluhm))
+  - Unit test fixes for python 3.9 [\#1858](https://github.com/hyperledger/aries-cloudagent-python/pull/1858) ([andrewwhitehead](https://github.com/andrewwhitehead))
+
+- Release management pull requests
+  - Release 1.0.0-rc0 [\#1904](https://github.com/hyperledger/aries-cloudagent-python/pull/1904) ([swcurran](https://github.com/swcurran))
+
 # 0.7.4
 
 ## June 30, 2022
 
+> :warning: **Existing multitenant JWTs invalidated when a new JWT is
+generated**: If you have a pre-existing implementation with existing Admin API
+authorization JWTs, invoking the endpoint to get a JWT now invalidates the
+existing JWT. Previously an identical JWT would be created. Please see this
+[comment on PR \#1725](https://github.com/hyperledger/aries-cloudagent-python/pull/1725#issuecomment-1096172144)
+for more details.
+
 0.7.4 is a significant release focused on stability and production deployments.
 As the "patch" release number indicates, there were no breaking changes in the
 Admin API, but a huge volume of updates and improvements.  Highlights of this

DIDResolution.md

@@ -29,7 +29,7 @@ In practice, DIDs and DID Documents are used for a variety of purposes but espec
 
 ## `DIDResolver`
 
-In ACA-Py, the `DIDResolver` provides the interface to resolve DIDs using registered method resolvers. Method resolver registration happens on startup through the `DIDResolverRegistry`. This registry enables additional resolvers to be loaded via plugin.
+In ACA-Py, the `DIDResolver` provides the interface to resolve DIDs using registered method resolvers. Method resolver registration happens on startup in a `did_resolvers` list. This registry enables additional resolvers to be loaded via plugin.
 
 #### Example usage:
 ```python=
@@ -73,17 +73,17 @@ The following is an example method resolver implementation. In this example, we
 
 ```python=
 from aries_cloudagent.config.injection_context import InjectionContext
-from aries_cloudagent.resolver.did_resolver_registry import DIDResolverRegistry
+from ..resolver.did_resolver import DIDResolver
 
 from .example_resolver import ExampleResolver
 
 
 async def setup(context: InjectionContext):
     """Setup the plugin."""
-    registry = context.inject(DIDResolverRegistry)
+    registry = context.inject(DIDResolver)
     resolver = ExampleResolver()
     await resolver.setup(context)
-    registry.register(resolver)
+    registry.append(resolver)
 ```
 
 #### `example_resolver.py`

Endorser.md

@@ -1,7 +1,5 @@
 # Transaction Endorser Support
 
-Note that the ACA-Py transaction support is in the process of code refactor and cleanup.  The following documents the current state, but is subject to change.
-
 ACA-Py supports an [Endorser Protocol](https://github.com/hyperledger/aries-rfcs/pull/586), that allows an un-privileged agent (an "Author") to request another agent (the "Endorser") to sign their transactions so they can write these transactions to the ledger.  This is required on Indy ledgers, where new agents will typically be granted only "Author" privileges.
 
 Transaction Endorsement is built into the protocols for Schema, Credential Definition and Revocation, and endorsements can be explicitely requested, or ACA-Py can be configured to automate the endorsement workflow.
@@ -61,3 +59,44 @@ Endorsement:
                         For Authors, specify whether to automatically promote a DID to the wallet public DID after writing to the ledger.
 ```
 
+## How Aca-py Handles Endorsements
+
+Internally, the Endorsement functionality is implemented as a protocol, and is implemented consistently with other protocols:
+
+- a [routes.py](https://github.com/hyperledger/aries-cloudagent-python/blob/main/aries_cloudagent/protocols/endorse_transaction/v1_0/routes.py) file exposes the admin endpoints
+- [handler files](https://github.com/hyperledger/aries-cloudagent-python/tree/main/aries_cloudagent/protocols/endorse_transaction/v1_0/handlers) implement responses to any received Endorse protocol messages
+- a [manager.py](https://github.com/hyperledger/aries-cloudagent-python/blob/main/aries_cloudagent/protocols/endorse_transaction/v1_0/manager.py) file implements common functionality that is called from both the routes.py and handler classes (as well as from other classes that need to interact with Endorser functionality)
+
+The Endorser makes use of the [Event Bus](https://github.com/hyperledger/aries-cloudagent-python/blob/main/CHANGELOG.md#july-14-2021) (links to the PR which links to a hackmd doc) to notify other protocols of any Endorser events of interest.  For example, after a Credential Definition endorsement is received, the TransactionManager writes the endorsed transaction to the ledger and uses the Event Bus to notify the Credential Defintition manager that it can do any required post-processing (such as writing the cred def record to the wallet, initiating the revocation registry, etc.).
+
+The overall architecture can be illustrated as:
+
+![Class Diagram](./docs/assets/endorser-design.png)
+
+### Create Credential Definition and Revocation Registry
+
+An example of an Endorser flow is as follows, showing how a credential definition endorsement is received and processed, and optionally kicks off the revocation registry process:
+
+![Sequence Diagram](./docs/assets/endorse-cred-def.png)
+
+You can see that there is a standard endorser flow happening each time there is a ledger write (illustrated in the "Endorser" process).
+
+At the end of each endorse sequence, the TransactionManager sends a notification via the EventBus so that any dependant processing can continue.  Each Router is responsible for listening and responding to these notifications if necessary.
+
+For example:
+
+- Once the credential definition is created, a revocation registry must be created (for revocable cred defs)
+- Once the revocation registry is created, a revocation entry must be created
+- Potentially, the cred def status could be updated once the revocation entry is completed
+
+Using the EventBus decouples the event sequence.  Any functions triggered by an event notification are typically also available directly via Admin endpoints.
+
+### Create DID and Promote to Public
+
+... and an example of creating a DID and promoting it to public (and creating an ATTRIB for the endpoint:
+
+![Sequence Diagram](./docs/assets/endorse-public-did.png)
+
+You can see the same endorsement processes in this sequence.
+
+Once the DID is written, the DID can (optionally) be promoted to the public DID, which will also invoke an ATTRIB transaction to write the endpoint.

actions/run-integration-tests/action.yml

@@ -20,9 +20,12 @@ runs:
     - name: run-integration-tests-acapy
       # to run with external ledger and tails server run as follows (and remove the ledger and tails actions from the workflow):
       # run: LEDGER_URL=http://test.bcovrin.vonx.io PUBLIC_TAILS_URL=https://tails.vonx.io ./run_bdd ${{ inputs.TEST_SCOPE }}
-      run: LEDGER_URL=${{inputs.IN_LEDGER_URL}} PUBLIC_TAILS_URL=${{inputs.IN_PUBLIC_TAILS_URL}} ./run_bdd ${{ inputs.TEST_SCOPE }}
+      run: ./run_bdd ${{ inputs.TEST_SCOPE }}
       shell: bash
       env:
+        LEDGER_URL: ${{ inputs.IN_LEDGER_URL }}
+        PUBLIC_TAILS_URL: ${{ inputs.IN_PUBLIC_TAILS_URL }}
+        LOG_LEVEL: warning
         NO_TTY: "1"
       working-directory: acapy/demo
 branding:

aries_cloudagent/admin/server.py

@@ -4,7 +4,7 @@
 from hmac import compare_digest
 import logging
 import re
-from typing import Callable, Coroutine
+from typing import Callable, Coroutine, Optional, Pattern, Sequence, cast
 import uuid
 import warnings
 import weakref
@@ -261,9 +261,32 @@ def __init__(
         self.websocket_queues = {}
         self.site = None
         self.multitenant_manager = context.inject_or(BaseMultitenantManager)
+        self._additional_route_pattern: Optional[Pattern] = None
 
         self.server_paths = []
 
+    @property
+    def additional_routes_pattern(self) -> Optional[Pattern]:
+        """Pattern for configured addtional routes to permit base wallet to access."""
+        if self._additional_route_pattern:
+            return self._additional_route_pattern
+
+        base_wallet_routes = self.context.settings.get("multitenant.base_wallet_routes")
+        base_wallet_routes = cast(Sequence[str], base_wallet_routes)
+        if base_wallet_routes:
+            self._additional_route_pattern = re.compile(
+                "^(?:" + "|".join(base_wallet_routes) + ")"
+            )
+        return None
+
+    def _matches_additional_routes(self, path: str) -> bool:
+        """Path matches additional_routes_pattern."""
+        pattern = self.additional_routes_pattern
+        if pattern:
+            return bool(pattern.match(path))
+
+        return False
+
     async def make_application(self) -> web.Application:
         """Get the aiohttp application instance."""
 
@@ -336,6 +359,7 @@ async def check_multitenant_authorization(request: web.Request, handler):
                         path,
                     )
                     or path.startswith("/mediation/default-mediator")
+                    or self._matches_additional_routes(path)
                 )
 
                 # base wallet is not allowed to perform ssi related actions.