I'll answer my own question to tell you how I solved this problem. It might help others to check stuff like this before jumping into conclusions.
Yes, it was a cooling problem. I found that a motherboard fan connector was loose. Fans were working but when the CPU was overheating, that information wasn't passed to fans. When I plugged that connector back, temperatures got back to normal.
