You must log in or register to comment.
A coworker once got an HR talking-to for printing this meme out and leaving it on all the dev’s desks.
As a data engineer, testing with production loads is critical to performance checking, as well as finding edge cases where your assumptions about what can be expected in the data are curb stomped and send you back to the drawing board to cry and think about what you’ve done.
Yeah we finally set up a workflow where we get production data available in a staging environment. This has saved a lot of trouble via “well it worked on my local where there were 100 records, but prod has 1037492 and it does not”
And then managers that don’t get this will try to shove policies down our throats about how “pre-prod systems should not have access to prod data”.
“just obfuscate it.” Sure, for all 300Tb of it from the 10 different sources that don’t really talk to each other and we were already doing magic to be able to join them together? They should give us a bottle of hard liquor per month/project.
