I know rankings are not affected if you don't have a responsive design, so it still isn't a crucial thing and visitors to a website don't care about it, they just want to be able to see the content. It's pretty cool to have a responsive design, though.