To me, this is as important as making characters itself. How your world is structured would make your characters, it would show what kind of society they grew up in, the kind of beliefs they were raised in. And to me, it is important.
Some writers treat this as a character itself, to me it is an interesting perspective to see. That the world is intricate and hard to define in one sentence, it basically sums up our world. Where there are so many things which are so simple yet so complex, there are completely different explanations for children and adults. Just like morality, Jo answer is completely right, and most of the time it comes in shades of grey.
To me, history plays an integral part in shaping my world. It tells me that to make a believable world, I need to show the history having its own merits and faults, and how the world is flawed itself. If the world was perfect, it would be called an utopia rather than what it is. And writing utopian works; they are simply hard to pull off. How do you write a work of an utopian world, a dystopia would work far better.
To me, it is also figuring out how the world would impact the characters and what sort of characters might come out of it. Especially in long sagas about fantasy, the world plays a large part in making characters who they are or what they believe. Such as whether it’s a patriarchal society or matriarchal society, whether both genders are treated equally, whether they respect power and authority. It all plays a part in making the characters.
With the world set, possible back stories can be made with the context of the world itself, and hence making the character even more believable with the environment they have grew up with and the sort of life they had lived. As long as the world is developed enough to justify the personality of the character and also how it happens to them.
To me, worldbuilding can also be a fascinating topic. I once did a note on nothing but earrings, but they were quite an important way to show their status in my book, as such the development was necessary to me. For me, worldbuilding is about as much as the important beliefs, traditions, more than miniscule details which might matter little. For me, focus on what is important in the world. And don’t be afraid to use the real world as an inspiration and helping you to figure out some details and also use objects as a way of symbolism. That to me, has always been one which I loved to use in my books if I could find an appropriate form to use it.
Worldbuilding can be fun, and also sometimes just let yourself loose and surf the net. I always end up surprised what I end up searching, the history of the world is basically your best friend here. There are plenty of interesting facts about the world waiting to be discovered in fact.
As for those who didn’t worldbuilding daunting, just stop trying to fill your notes with words and details that might matter little, more on what is important and vital to the story for it to feel realistic. And also, leave it alone until you feel as though you want to add something. Mine are still sparse till this day but I keep in mind on what I develop and write in the actual document itself.