The time length and the fact you don't really gain something permanently by yourself is a direct proof. The fact that this manifests in this way it's because of their energy. When this may wear off after some time (which can take a lot of time) this may go away. Also, it will not leave you with permanent gifts or anything like this, which you will have irrespective of summoning or not summoning, which one would have if they had this on their own.
As you advance this would be clear.