Ah yes - I know exactly what you are talking about and I am more or less able to personally bypass it - however you have some very interesting points there; tools - situation specific matters - survival - strategy; some of which the general mechanism behind Bounded Rationality could be good at. Just change out the rationality for whatever else.
Further Regarding AI Domination: I think that AI is already changing us just like other technology - it is just that most of us do not really get that yet - but there is plenty of weak AI around already in smart phones, automobiles, process control systems and hell, google is a form of AI. I think it is more likely that as we build AI and it changes and enhances us and our lives that our understanding of it will increase and we will put that new understanding into AI which will further enhance us and our lives and so on and so forth et cetera et cetera - I mean take your pick of philosophies there - I think even cause and effect offers us some insight here.
An aside: Loved The Matrix - Loved I Robot; The movie Arrival even though not really about AI is great and shows insight in many realms - for me the what if we made first contact with aliens correlated with first contact with AI has been and gone without any of us noticing - but what if first contact with AI was more like an alien first contact rather than like a bacteria or basic organism like it was. Excuse the type of writing I use in this particular paragraph, it is designed a particular way to stimulate thought but you may have to read it again.
I like your idea of self-awareness regarding personal conduct and in this way I think an advanced AI will be able to do this easily - think Data from Star Trek - The Next Generation.
But movies and television aside - with some deeper thought one can see the benefit and given we have been considering the risk for so long - I have this strange faith in humans to make and do the right thing. Mostly people think that one day we will wake up and there it is, first contact - not a chance, this will be gradual transitions to very smart machines even if somewhat quicker than before the transitions will smooth out the faster we develop - I have already seen this in experiment.
We have good studies, knowledge, wisdom and other gems like self-reference effect. Indicating artifacts like “influence”.
And some of the mathematics we use these days - the good stuff for AI - people have not even discovered yet - and yet simple mechanisms in code like TRUE, FALSE, AND, OR, XOR, NOT, NAND, NOR, and XNOR serve well for basic truth, logic, error checking and with a bit of imagination who knows.
An aside:
An idea of the type of mathematics that gets considered around my AI projects: Fractals, Space-filling curves, Quasigroups, Disordered hyperuniformity, Groupoids, Quasicrystals, 5-cell; Just to name a few - the ones I find interesting - the ones that spring to mind anyway.
So back to it - I am wondering whether there are infinite ways to build intelligent machines - whether we could actually model all the theories through out history as computer simulated intelligence. Imagine a Kant machine - although I often wonder whether he was already a philosophy machine. Imagine a YOU machine or a ME machine - the possibilities are endless. The system I am working on now grows with the user - it starts out with nothing at all - it attempts to model the text world around it similar to the way we do. So the more you talk to it the more it knows. If we consider all layers in philosophy and think about the concept of layers we find ourselves thinking in layers - nature did this with the Neocortex - the Neocortex has six layers and many columns and then in humans it decided to fold itself to allow for more surface area - this bending, repeating, asymmetric, multidimensional plane makes no sense really but just thinking of its geometry makes you think of geometry - geometry is so easily correlated with algebra. This correlation allows us to impose multidimensional objects onto a plane and then layer these planes - if you are still following me at this point then you can see the implications but if you are not then never mind because your brain is already doing it for you.
The last paragraph sounds a bit nuts in hindsight but it holds many insights. So what are we talking about? A bit of everything for now.