Can you ever become stateless after becoming Japanese?
Statelessness is the condition where a person has no legal nationality associated with them.
The Japanese Constitution, Article 22, specifically grants Japanese nationals the right to lose Japanese nationality by choice:
Freedom of all persons to move to a foreign country and to divest themselves of their nationality shall be inviolate.
Despite this, however, it is currently impossible for a Japanese national to simply give up their only nationality and become a person with no nationality.
This is because Japan is a signatory to the 1961 UN Convention to reduce statelessness. The Japanese Constitution was written before this, and Article 98 of the Constitution says that Japan must obey the UN Conventions and treaties that it signs:
The treaties concluded by Japan and established laws of nations shall be faithfully observed.
Thus, the only way to lose Japanese nationality is if you already have another nationality to fall back on.
What if it's discovered that one's naturalization process was fraudulent?
As long as you only possess Japanese nationality and no other nationality, Japan must abide by United Nations conventions and not strip you of your Japanese nationality and make you stateless. Even if you somehow lied and fooled Japanese authorities into giving you Japanese nationality when you shouldn't have been able to acquire it.
In a situation such as this, most likely the Japanese government will prosecute you for perjury or fraud against the government, which may result in a punishment like a fine and/or imprisonment if the perjury is considered to be severe enough.
Now, if you possess another nationality, that's a different story: Japanese (or any other) nationality can be removed according to international law if one possesses another nationality and the laws of that country permit it.
The United States is the outlier
Almost all western European countries and other advanced developed democracies that follow United Nations conventions are just like Japan in that they will not let you become stateless.
America, however, is the exception. Though American embassy officials do not recommend it and they will try very hard to talk you out of it, it is possible to make yourself stateless if you are an American. The reason Americans do this is usually as a form of political protest or because of a disbelief in the world's nation-state system. Becoming stateless does not relieve or excuse one of their outstanding debts, taxes, or crimes.
Eliminating Statelessness is why Japan has jus soli
Nationality in Japan is normally inherited from the legal nationality of the parents: if one of the parents has Japanese nationality, then the child gets Japanese nationality.
In contrast, most of the countries in North and South America follow the principle of jus soli, which means a newborn baby inherits the nationality of the territory they are born on.
However, Japan does have one exception where they allow babies to become legally Japanese simply by being born on Japanese soil: if the parents are stateless, or if the nationality of its parents are unknown (probably because the parents are deceased) or for some reason the baby can't inherit the nationality of its parents.
Japan carved out this exception in its nationality law to comply with UN Conventions aimed at reducing and eliminating statelessness.