Data trusts handle data in a fiduciary capacity for data owners, allowing them to process, aggregate and share data with other stakeholders within an overarching legal and ethical framework. One of the primary challenges of data trusts is consent management. This paper characterizes the problem of consent management in data trusts and proposes a 4-layer architecture for a policy-based domain-agnostic consent management service, meant to operate within one or more regulatory frameworks. We address architectural questions of how to make data access legitimate, by providing a policy interpretation for each access, and how to prevent the data trust itself from accessing data that it stores, by overriding access policies.