First thing one should know is that everything in this world has a counterpart in the spiritual worlds. For example, during the plague of darkness in Egypt, it says the darkness was so strong that the Egyptians could not move for 3 days (Exodus/Shmos 10:23). How does darkness prevent one from moving?? The answer is they were given a taste of the corresponding darkness in the spiritual worlds which is a kind of total confusion and disorientation. For this they couldn't move. At the same time, the Midrash says the Jews had light and could see where the Egyptians hid their gold. How? Because, they were given a taste of the corresponding "light" in the spiritual worlds which is a kind of supernatural, clairvoyant light (from Ner Uziel). This light is hidden in the torah. One who toils properly in the talmud, even though the subjects seem completely irrelevant, nevertheless this light has a powerful spiritual effect on a person and can make him into something special. This is why very great torah scholars possess supernatural, clairvoyant powers. However the condition is that one toils in the way of chizuk (strengthening and exertion) and mesirus nefesh (intense effort) for several years.
Here's an excerpt from Rabbi Moshe Chaim Luzatto, considered the greatest torah scholar and kabbalist in the past few hundred years.
Excerpt from Derech Etz Chaim by Rabby Moshe Chaim Luzatto
The creator of man, and his guide, made him and established him ready to understand and to comprehend greater comprehension than the ministering angels. And the Sages have already said (Midrash Bereishis Raba 17):"He (G-d) said to them (the angels): 'his wisdom is greater than yours'." And when man desires understanding in G-d's way, G-d will guide him to fathom until the heaven of heavens - things which stand in the loftiest heights of the universe. But, behold, the good is in his hands and he has the power to choose - to become wise and to know, or to remain naked of all wisdom, even though the heart and the intellect are inside his body.
And this you will see, that there are two things which were created with one likeness (both function together) - the intellect of the man and the torah which gives him the intelligence. On the torah it is written "Is not My word like fire? says the L-ORD; and like a hammer that smashes the rock in pieces?" (Yirmiyahu 23:29). He has told us in this that it is the truth of the matter that the torah is literally a light that was given to Yisrael (the Jewish people) to enlighten in it. Because it is not like the wisdom of the nations and the secular knowledge, which are nothing more than knowledge which the intellect reaches after toiling (to understand it). Rather, the torah, behold, it is holy, which has a supernal existence in the loftiest heights, and when a man toils in it below, it is a light which illuminates in his soul to elevate it to the treasures on high, the treasures of the Creator, may His name be blessed. And this is what the wise man said (King Solomon) "And torah is light" (Mishlei 6:23), literally light, and not just wisdom. And not that it is called "light" as a kind of comparison rather it is literally "light". Because this is its existence above (in the higher worlds), and when it enters the soul, the light enters it just like the sun's rays enter inside a house.
Behold with great precision it (the torah) was compared to fire and with precise choice of word. Because when you see an ember which does not flame, and the energy of the flame is inside, hidden and closed, until one blows on it, then the flame spreads out and goes forth, and in that flame is visible many colors, what was not previously visible in the ember, and everything came out from the ember. So too, the torah which is in front of us, because all of its words and letters are like embers, which when you ignite them look the same. One sees only embers which are also almost dim. And he who exerts himself to toil in it, then it will flame from every letter a mighty flame full of many colors, which are the knowledge which are hidden inside that letter. This has already been explained in the Zohar on the Alef-Beit, and the matter is not an analogy, rather it is itself actual and completely literal. Because all the (22) letters that we see in the torah, all of them teach on the 22 lights which exist on high. And these supernal lights, shine on the letters. And from them comes the holiness of the torah, the holiness of the sefer torah, tefilin, and mezuzos, and all the holy writings. And according to the holiness in which they were written, so too will be the resting and shining of these lights on the letters. Therefore the sefer torah which has one invalid letter, becomes entirely invalid because the lights cannot remain in it properly, that it should pull down the holiness to the congregation with the power of the reading of it.
Returning to the matter, the lights which stand on the letters contain all the levels and sub-levels which are in each letter. However, it doesn't reach the soul of the one who sees the letters except a small, concealed light, like an ember. But when the man exerts himself to understand, and he reads, and returns to read, and strengthens (mitchazek) himself to understand, behold, the lights ignite and go out like the flame from an ember. And on this the Tana (Mishnaic Sage) said "Turn it and turn it again, for everything is in it" (Avos 5:21). Because the one who toils needs to turn it over again and again until it flames exactly like the actions for [physical] fire. And behold when the flame ignites, I've already mentioned that there are in it (in fire) many colors and forms, and likewise there are found many great matters in the flames of that fire (i.e. fire is a complex phenomena, with chemical energy and atomic radiation of visible light due to the heat). However there is another matter that we find, namely that there are many sides (faces, levels of understanding) to the torah. And the early ones already received (from Sinai) that for each root of the souls of Israel they are in the entire torah, therefore there are 600,000 explanations of the torah (corresponding to the 600,000 root souls). And this is what is meant by the torah "splits to many sparks", because at first it flames and then all the lights in it become visible on that matter. And those lights, themselves, enlighten to 600,000 ways in the 600,000 (souls) of Israel. And this is the secret meaning of "Is not My word like as fire? says the L-ORD; and like a hammer that smashes the rock in pieces?" (Yirmiya/Jeremiah 23:29)
It comes out from this, that even though the torah is endless, and even each letter from it is like this, however one needs to ignite it, and then it flames. For this, the intellect of man was made. Because it also has the power of great attainment, but when it flames (the intellect of man) it is through the power of contemplation. And on this it is written "For the L-ORD gives wisdom, out of His mouth comes knowledge and understanding" (Mishlei 2:6). Because all that exists from the word of the Holy One blessed be He, becomes existing (i.e. has an existence), because this "mouth" (of G-d) is the root of all creations, and this itself sustains their existence. And the "hevel" (lit. breath, or vapor which exudes from the mouth), which means the hashpaah (influence, energy) which goes out to all creations form the source which took it out. And therefore it is written "that man does not live by bread only, but by every thing that goes out of the mouth of the L-ORD does man live" (Devarim 8:3), which is this hevel which goes out and sustains. And this (hevel) envelops and enclothes itself inside the edible foods to nourish a man.
Behold, the wisdom is already given from the Holy One blessed be He in the heart of every man. However, in order that it strengthens, it is necessary that this mouth (i.e. the mouth of G-d) that sustains its existence should blow with strength, and then it also becomes like fire, that when one blows in it, it flames. Likewise when this hashpaah (influence, energy) descends from the mouth like a breath, the wisdom flames and the knowledge and understanding which were inside it already (i.e. inside the intellect of man) become visible, just that they were not visible before. Because there was nothing visible before except the chachma which is the intellect itself, but the tevuna (understanding), which is the understanding of the intellect which joins things together (logically) , and upholds things together, and the (daas) knowledge which is the offspring (result) which comes out of this - and these won't do their work except through the power of the blowing of the breath which that supernal mouth blows on them. Therefore, we find, that the wisdom (chachma) is already given, and the mouth only sustains it, but the understanding (tevuna) and the knowledge (daas) they renew, which means their revelations are renewed all the time through the blowing of that mouth. And therefore Elihu said "But it is a spirit in man, and the breath of the Almighty, that giveth them understanding" (Iyov/Job 32:8), which means the breath of the mouth, the breath of the Ruach (wind) and this is what gives a man understanding - and not days or years.
Therefore it is a duty on a person to put himself to contemplate. Because if he does not contemplate and exert himself to think, behold, the wisdom will not come to seek him, and he will remain in darkness without knowledge, and he will go in the ways of vanity and obscurity and the end of the matter is that he will give judgment and accounting before the King of kings, the Holy One, blessed be He on that he did not use the wisdom (chachma) and the power which He implanted in him, [and instead] made it profane. This is certainly the temptation of the yetzer (evil inclination) which tries with all its might to make him fall in this world and the next, because he knows already that if he falls in this a little bit, it remains (difficult) for him to return and rebuild, because the knowledge of truth strengthens the soul and repulses the yetzer (evil inclination), and there is nothing that weakens the soul before the yetzer like lack of knowledge. And if the knowledge was wide and firmly standing in the hearts of men, they would never sin. The yetzer would not even be close to them and rule over them, just like he does not rule over the angels.
And being that the Holy One blessed be He wanted that man should have a yetzer, that he be capable to victorious or defeated with equal weight. Therefore He made in them the knowledge, but it is closed like an ember, although capable of exanding and flaming like a fire. And the free will is in man's hands. Our Sages of blessed memory said "and therefore those who rule over their yetzer say 'Come, let us make the calculation of [how to act in] this world'" because he who does not rule over his yetzer (evil inclination) he will not put this to himself ever. But those who rule over their yetzer, they do these things and teach it over to others so that they may do it themselves. Behold a man, most of his years are spent in thinking thoughts on his businesses, business of this temporary world. Why does he not put to heart even one hour also on thinking these other things - what is he? Why did he come to this world? Or what does the King of kings seek from him? What will be the end of his matter? This is the biggest and most powerful medicine that one can find against the yetzer. It is easy and its effects are great. Its fruit is many, in that a man should stand each day for at least an hour, free from all other thoughts, and to think only on this matter that I've said. And he should seek in his heart:
* what did the early ones, the fathers of the world, that G-d desired in them? What did Moshe Rabeinu do? What did David, the moshiach (chosen of) H-shem do? And all the gedolim (great men) who lived before us? And he should put to his mind -
*what is good for a man all the days of his life in order that he should also do the good. Then he should explore with his thoughts to know what is his situation that he is in and that he stands in relative to that which is wanted [by G-d], in the path which trod these men of G-d before him.
(translator's note: these are things one must work out and verify on his own. It's not enough to read answers from a book. If a person does this truthfully, he will see how unclear he is about so many of these fundamental life-deciding issues).
And if not? And if the power of the yetzer is too strong and he cannot bring himself to judge due to its strength - behold for this Rabbi Shimon trembled and screamed like a rooster in his book - "Voy to the men who G-d has abandoned in the exile - as in 'And he looked this way and that way, and when he saw that there was no man' (Shmos/Exodus 2:12), rather each man turns to his ways, to his business, to his path."
The general principle: The man who does not think on this, it is very, very hard to reach the shlemus (wholeness), and the man who does think - he is very close to it. (the great men, )They would not stray their minds from this, therefore they succeeded in all their actions, but at least one should do the minimum and fix times, to put to his heart whether a little or a lot - because then he will succeed in his ways and then he will understand
After you have known this, that this charge is on you to contemplate, and to comprehend. Now I will make known to you and enlighten you what are the things on which you should stand on them to contemplate and explore.
Behold, the first investigation which you should investigate is on His intention, may His name be blessed. For what, and why did He create all of this great creation. And don't say "what is man to come after the King to seek His thoughts? The verse testifies 'Your thoughts are very deep' (tehilim 92:6). And this would seem like arrogance and high-heartedness to say what you can find the intention of the Creator!"
Just the opposite, on this it says "the secret of G-d is for those who fear Him" (tehilim 25:14)...(see the hebrew for more)
....
(skipping forward, after some heavy kabalistic discourses...)
Behold from these discourses, you will understand with your pure intellect that there is no medicine for our wound (after the investigations mentioned above) except the toil in the torah. Because there is nothing that weakens the sitra achera (side of evil) except the toil in torah.....(see hebrew for more)
click for the original hebrew of Derech Etz Chaim
for further reading, see the classic Path of the Just by Rabbi Moshe Chaim Luzatto, section "watchfulness"
There he starts off the book saying:
"The foundation of piety and the root of perfection in the service of G-d lies in a man's coming to see clearly and to recognize as truth the nature of his duty in the world and the end towards which he should direct his vision and his aspiration in all of his labors all the days of his life..."
Note that Rabbi Luzatto compares one's clarity of his duty in the world to a "root". What's the comparison? The answer is, the root of a tree is its lifeforce. So too the clarity of a person is the lifeforce of his service. If his clarity is weak, his lifeforce and motivation is proportionately weak. The Chofetz Chaim would arise almost every night in solitude and and for some time play out as if he died and went on to the next world. I also read about an extremely wealthy Jew who would lie down in his future grave daily for some time to remind himself of what's really important in life.
