--

Hi Jonas, it depends a bit on how you design and process data, but overall I recommend to already inherit all datatypes in Bronze. In Bronze, you usually create new files, so you won't have any problems yet. Consequently this also means that you need to validate data before you move it into Silver. So, in that scenario I would write a bunch of test scripts (DBT test) to validate all data in Bronze. So are datatypes what you are expecting, are fields not nullable, unique, etc. All these tests you should do upfront, before processing and integrating with Silver.

--

--

Piethein Strengholt
Piethein Strengholt

Written by Piethein Strengholt

Hands-on Chief Data Officer. Working @Microsoft.

No responses yet