Reproduced the issue, and saw "try1" instead of the expected "try3".
Replaced the fileEntry with an h:inputText with required=true, and go "try3" as expected, showing that the issue is dependent on the fileEntry.
Added the fileEntry back in, along-side the h:inputText, and didn't select any files to upload at all, continuing to use the h:inputText validation. Got "try3" as expected", indicating that it's not the alternate ajax mechanisn used when a fileEntry is present, but rather something about how fileEntry validation affects the lifecycles.
Looked at the requests and responses. See that on try1 (valid fileEntry), we get back textarea display:none, script to render the editor, the file upload success message, and the clearFileSelection script. With try2 (invalid fileEntry) we get back the textarea display:none with no editor rendering script, the file upload failure message, and no clearFileSelection script. On try3 (valid fileEntry), "try3" is not actually sent but rather "try1" is, indicating that the lack of the editor rendering script in the invalid fileEntry lifecycle response is the root of the problem.
Note that fileEntry actually does its validation early be default due to backwards compatibility. Now setting immediateValidation="false". This actually worked. Interestingly, for try2 and try3, the rich text editor script came back in the dom diff but not the textarea. applyDomChanges explains the no textarea in response, and in theory if the rich text editor used validations phase to determine to render the script instead of render phase, that would explain the bad behaviour with fileEntry's default immediateValidation="true".
Verifying by ui:remove the fileEntry again and adding immediate=true on the h:inputText. Same problem, that the try2 response contained "try1", indicating some issue with the rich text editor not using a submittedValue, and no editor script, indicating it being decided to be rendered not in render phase but in validation phase.
So, this is not a fileEntry issue at all, but solely an ice:inputRichText issue with not functioning after another input component fails validation when having immediate="true", whereby validation happens in the decode phase.
Attached is a webapp project (not a portlet) that demonstrates the problem.
Steps to reproduce:
1. Deploy the WAR to the tomcat/webapps folder
2. Visit the following URL: http://localhost:8080/ice-8474-test-1.0.0/views/test.xhtml
3. Enter "try1" in the rich text area, and select a file less than 1MB with the ace:fileEntry
4. Click on the "Add Attachment" button
5. Verify that you see a good message in the browser, indicating that the file upload was successful
6. Verify in the Tomcat log that you see the following:
ModelBean.setComments(String) comments=[<p>try1</p>]
7. Enter "try2" in the rich text area
8. Select a file that is greater than 1MB with the ace:fileEntry
9. Click on the "Add Attachment" button
10. Verify that you see a bad error message in the browser, indicating that the file was too big
11. Enter "try3" in the rich text area
12. Select a file that is less than 1MB with the ace:fileEntry
13. Click on the "Add Attachment" button
If the bug still exists, then you should NOT see the following output in the Tomcat log:
ModelBean.setComments(String) comments=[<p>try3</p>]
Instead you will see the wrong value. Either an older one, or blank.