I have some data that suggests the affection you gain from talking to a guy increases approximately exponentially with your level in the hobby for the guy's preferred stat, but I'm having trouble finding the precise formula.  What is evident is that the hearts for talking depends only on the hobby level.  Maybe it'll be easier to deduce the correct formula working collaboratively, so I'm including the data I've gathered so far (which is up to lvl 32 hobbies).  Hopefully tables aren't broken in forum posts...

Before I get to that, though, I have determined to a fairly high degree of precision that the affection gained by poking is 1/16 of the affection gained by talking, rounded to the nearest integer.  By the time I get to lvl 32 hobbies, the auto-generation of hearts goes by too fast for me to see the individual hearts, but the tens place and up are still observable and match perfectly.

My best approximation so far isn't great, but it's a starting point:

(1.565)^(hobby level + 5)