Can you not renounce your original citizenship and hide this from the government(s)?

In the case of the U.S., there is no need to hide additional citizenships, acquired involuntarily or voluntarily. The U.S., due to its history with slavery and its response with the 14th Amendment, has made it almost impossible for the U.S. to revoke somebody's properly acquired citizenship, no matter what you said or did or promised to a foreign government like Japan. Even if you did something so bad that you'd be a candidate for denaturalization, the modern U.S. government would simply throw you in jail (or execute you via the death penalty), rather than revoke your U.S. citizenship.

Even so, a U.S. State Department page still warns that:

... U.S. citizens are subject to loss of citizenship if they perform certain specified acts voluntarily and with the intention to relinquish U.S. citizenship.

However, what the U.S. State Department says on its site and what the U.S. Supreme Court has ruled (overriding the original stricter intent of the Immigration and Nationality Act) and the State Department now actually does regarding "potentially expatriating acts" are very different things.

The combination of the 14th Amendment and its subsequent interpretations (i.e. Nishikawa v. Dulles, Afroyim v. Rusk & Vance v. Terrazas) means that even if you get conscripted to fight against the United States, vote in a foreign election, or swear an oath (verbally or in writing, to a foreign official, flag, king, emperor, or bible) to a foreign government and constitution that you intend to naturalize or renounce, it doesn't really count unless you actually formally and willfully report the expatriating act directly and in-person at a U.S. Consulate/Embassy intending to lose your U.S. citizenship.

In other words, there is a loophole on the U.S. side due to the 14th Amendment and the rulings on it: the U.S. essentially permits you to lie to foreign governments (on paper, verbally, to a flag, a Bible, or official) with respect to your intentions regarding your citizenship. What kind of trouble that gets you in with the other government is your own business, but the U.S. won't get involved just as long as you don't declare to an authorized U.S. official at an embassy in person that you intended or intend to lose your U.S. citizenship.

Some U.S. politicians don't like this, but they know that overriding the Supreme Court requires either a new Constitutional Amendment or the Justices to overturn precedent — both extremely difficult propositions, so various locales (even at the federal level with H.R. 3938, which almost made it into law in 2005) instead try to "neuter" additional citizenships: that is, they make it illegal to exercise other citizenship's privileges (voting, holding office, etc — with perhaps the exception of border entry/exit for immigration rule compliance) while being a U.S. citizen, effectively making the additional nationalities "honorary" with respect to most of their citizen's privileges.

---

On the other hand, the Japanese Constitution has no such equivalent to the 14th Amendment, and while there are no penalties, financial or incarceration, for breaking the naturalization requirements regarding renouncing/relinquishing/abandoning your other citizenships, the Japanese government can strip you of your naturalized Japanese citizenship if you "forget" to renounce your other citizenship(s). The process is called "administrative denaturalization," and is basically done when someone commits fraud or fails to fulfill all the requirements of naturalization or voluntarily (not automatically or involuntarily) acquires another citizenship as a Japanese.

The U.S. can also strip people of their U.S. citizenship if (and only if, due to the 14th Amendment) that citizenship was acquired through a fraudulent naturalization application.

BUT if you only have one citizenship, be it Japanese or U.S. or any other country, they cannot strip you of it, as per U.N. Universal Declaration of Human Rights Article 15:

1. Everyone has the right to a [one] nationality.
2. No one shall be arbitrarily deprived of his nationality nor denied the right to change his nationality.

(Most countries try to never make or let a person become stateless. Some countries, such as the U.S., make it close to impossible to become even temporarily stateless, which is why the American procedure is to renounce after naturalizing to Japanese)

Once stripped of your Japanese citizenship, you'd have to apply for a visa to continue to work/live or even visit as a tourist. It's not clear in this case whether one would be once again be able to obtain a normally acquirable Japanese visa (even a temporary landing permit aka no-application-necessary "tourist visa"), as you'd have a mark on your permanent record indicating you committed fraud with the Japan Ministry of Justice, which controls Japanese Immigration.

That being said, administrative denaturalization has never occurred under modern Japanese nationality laws (1950 onwards). Nobody knows why, but the most common theory is that in the case of naturalization, there are not (yet) enough cases of naturalization abuse to make it a priority for enforcement. Furthermore, there are few egregious cases of dual-citizenship privilege abuse (for example, attempting to vote twice — which is by definition undemocratic — in two countries on the same issue such as an international trade or military agreement) to make the non-trivial administrative process worth it.

Considering that a person naturalized to Japan would most likely have their entire life (home, investments, career, family) in Japan, the remote possibility of an administrative denaturalization — which could mean being forced to leave Japan and abandon everything one has invested in it (financially, emotionally, and otherwise) if they couldn't get a Japanese visa after denaturalization — even if it's so rare it's unprecedented, should be enough of a deterrent to keep the sensibly risk averse from abusing the laws.

---

A dual-national by birth (for example, a Japanese-American born after 1985*) that has lived in the U.S. their whole life that fails to choose a nationality after 22 years of age does not usually have that upend-your-life denaturalization ramification that a naturalized citizen has, and they're more likely to try to get away with possessing multiple nationalities.

Japan requires that you use a Japanese passport to enter and leave the country if you possess one. And the U.S. requires that you use a U.S. passport if you possess one. And there are potential penalties if you fail to present the proper passport to either countries' immigration officials. Because of this, a Japanese official could figure out, by looking at the Japanese passport and/or the computer records, whether one has multiple citizenships by noticing the lack of stamps (U.S. visas, for example) and one's length of stay overseas. Furthermore, certain services for overseas Japanese (absentee voting, family register updates, etc.), require the presentation of a Japanese passport with the proper visas proving you're an ex-pat Japanese.

If a born-after-1985 over 22 years old Japanese-American is discovered at the Japan border with dual nationality, they are forced to choose as per post-1985 nationality law. If they choose the U.S., they are given a 国籍離脱届 {kokuseki ridatsu todoke} ([Japanese] nationality renunciation form), and their Japanese passport gets cancelled on the spot. If they choose Japan, they are told to renounce their other citizenships as soon as possible, since they can't realistically march them to that country's consulate/embassy. And even if they got their on-the-spot choice in writing, it wouldn't matter for Americans, again due to the Supreme Court's interpretation of the 14th Amendment. This is the loophole that some Japanese-Americans use to try to keep both nationalities if they are caught at the border. Nobody knows if or when the Japanese government will start keeping track of their choices and then denaturalize them (or tax them, which many Japanese-Americans would consider to be a harsher punishment!) upon discovery that they failed to renounce their foreign nationality.

Still, because of culture identity issues in at least America, possessing a CJK passport and by extension citizenship is a point of pride for many Asian-Americans, given the still strong non-legal association between ethnicity and nationality — even if they have no plans for ever living in Japan and they can't exercise their hyphenated citizenships overseas. Thus, most born-after-1985 over-22 Japanese-Americans do not take the risk of getting pulled into the auxiliary rooms at Japan's airport immigration and keep their Japanese citizenship hidden, even though they're legally required to present their Japanese passport.

* Prior to 1985, a multi-national Japanese-American, etc., that did not choose a nationality upon becoming an adult was assumed to have "chosen Japanese nationality by default." This created a loophole on the Japan side because the Japanese adult probably did not, as a minor, voluntarily choose their other citizenship(s) (ex. U.S.), so they became legally proper multi-nationals if the other country/countries permitted it. This loophole was closed in the revision to the Japanese Nationality Laws in 1985, but those Japanese born prior to 1985 were grandfathered in and remain legally multi-national.