Awkward situation regarding my neighbour - what would you do?

My husband is forever complaining about the noise our downstairs neighbour makes and to be honest, i don't think he's that loud. We live in a flat so there has to be a bit of give and take where noise is concerned. I seriously don't think it's a problem as he doesn't make that much noise but my husband have started arguing over this and it's ended up with him and i falling out. He phoned me the other night when he was out asking if the guy was making any noise. I could hear music but not that loud and it was only 7pm. I told husband this and he's now asked me to keep a noise diary which i just think is ridiculous - the people and traffic outside are louder than our neighbour. What's the best way for me to deal with this as i really don't want the hassle of rowing with my husband, but at the same time don't want to see my neighbour getting grief for no reason (he's a nice guy) Help!