r49552: modified mobi:camera to first detect if the getUserMedia API is available and launch the HTML5 camera, otherwise attempt to use Bridgeit, if not installed, then display the fallback file upload input; also added behaviour of hiding the toggle button when a picture is taken and showing it again when pressing the redo button; added workaround for Android devices to tap on the still image to start the camera video stream; and added the 'facingMode' attribute to specify to start with the 'rear' or 'front' camera

Testing notes: please test this on all supported browsers and devices. Note that the file upload fallback might take 10 seconds to show, because it's triggered by an onfailure callback on the bridgeit javascrpt, which explicity sets a 10-second timeout. This is the way it has always been.

r49563: added HTML5 recording support for mobi:microphone and gave it precedence over the use of Bridgeit

r49564: removed script that blocked the ui when Bridgeit was not detected now that the HTML5 microphone is available

Preliminary testing with ICEfaces 4 trunk r49564 using Tomcat 7 with and without SSL. On mobile devices bridgeIt was not installed during testing.

android 6.0.1
Non-SSL Camera - After pressing the Camera button a "Choose file" button is
presented. After pressing Choose File the native camera opens. A front or rear facing camera can be chosen with the native controls. When returning to the app the "Image captured" button has a spinning indicator on the right.The indicator continues to spin several minutes after taking the photo.
The photo is successfully uploaded.

SSL Camera - After pressing the Camera button a small window pops up with a front facing video image. There is a Take picture and a Cancel Button. There does not appear to be an option to take a rear facing photo.
The photo is successfully uploaded.

Non-SSL and SSL Camcorder - Insists that bridgeIt be installed, will not function without it.

Non-SSL Microphone - After pressing Record Audio a "Choose File" button is presented. After pressing Choose file a "Choose action" dialog was presented. The options there were Camera, Camcorder and Documents. No microphone option was available.

SSL Microphone - After pressing the "Record Audio" button a dialog asking for microphone access appears. After allowing access a small window pops up with recording controls. After making a recording and pressing the stop button there are "Redo Recording", "Keep recording" and "Cancel" buttons. While making the recording the timer stays at 0. After pressing start the time shows the length of the recording.
The recording is successfully uploaded and can be played back.

ios 10.0.2
Non-SSL and SSL Camera - After pressing Camera button a dialog is seen "Cannot Open Page, Safari can not open the page because the address is invalid". Through the modal background you can see the "Having Problems? BridgeIt may not be installed" dialog. After dismissing both dialogs there is a "Choose file" button displayed, instead of the Camera button. After pressing Choose file there are 3 options "Take Photo, Photo Library and iCloud Drive"

After choosing "Take Photo" the native camera is loaded with the ability to take front or rear facing photos. When returning to the app the "Image captured" button has a spinning indicator on the right.The indicator continues to spin several minutes after taking the photo.
The photo is successfully uploaded.
Attempting to take a second photo fails. After pressing the camera button the "Cannot open page" and "Having Problems" dialogs are again displayed. After dismissing the dialogs the camera or choose file buttons are removed from the page.

Non-SSL and SSL Camcorder Insists that bridgeIt be installed, will not function
without it.

Non-SSL and SSL Microphone - After pressing Record Audio a "Choose File" button is presented. After pressing Choose file a "Choose action" dialog was presented. The options there were Take photo or video, Photo Library and iCloudDrive. No microphone option was available.

FF 47 and Chrome 54
Non-SSL and SSL Camera - No issues with camera or file uploads. There is no option to toggle front or rear camera.

Non-SSL and SSL Camcorder - Not available in FF or Chrome browser.

Non-SSL and SSL Microphone - Demo appears to function properly. I am prompted to allow access to the microphone and can record and upload files successfully. I am not able to hear the sound recording playback. It may be an issue with my laptop as I believe the microphone is broken.

IE 11
Non-SSL and SSL Camera - After pressing the Camera button a "Sorry the camera command for bridgeIt is not supported on this platform" dialog is shown. After dismissing the dialog you can browse for an image. After choosing an image and pressing open, a browser console error appears and you can no longer use the demo.
SCRIPT5009: 'updateThumbnail' is undefined
File: eval code (76), Line: 101, Column: 9
If the browser console is not open while testing, the image can be successfully uploaded.

Non-SSL and SSL Camcorder - Not available in IE 11 browser.

Non-SSL and SSL Microphone - No prompted to allow microphone use, instead I can upload a pre-existing .mp3 file. Playback is successfully.

r49570: fixed issue with spinner not disappearing after selecting an image file as well as a related Javascript error; fixed issue in ios not being able to upload a second image on iOS; temporarily set 'supportsFacingMode' flag to always true for testing on Android

There's no microphone capture supported on iOS or Android in the same way images are supported, when using HTML input elements of the file type. Nothing else is available. More detailed information here:
http://stackoverflow.com/questions/25287112/record-voice-from-iphone-using-html5

Testing notes:

• Don't test mobi:camcorder, see explanation below.
• Test the toggle button on Android again. A temporary change was committed to make sure to always render the toggle button.
• As for IE11, the microphone demo is working well for me on SSL and on non-SSL. Please test on a fully updated Windows system.

Adding support for microphone recording wasn't as simple as capturing a still image, which is possible to do straight from the HTML5's canvas element's capabilities. In the case of stereo audio capture, it was necessary to record the left and right buffers, interleaving them, and then encoding of the data. For this it was necessary to use a third-party API to do this job, which uses a background thread. This library uses the MIT license.

Recording video and audio+video is even more complex and it's not fully supported at the moment in most major browsers.
https://developer.microsoft.com/en-us/microsoft-edge/platform/status/mediarecorder/

The MediaStream Recording W3C (or Media Recorder) specification is still undergoing changes as of recently.
https://www.w3.org/TR/mediastream-recording/

Edge only supports audio recording at the moment. Chrome supports video without audio. Android doesn't have full support yet. iOS doesn't support these APIs at all. Firefox and Opera seems to be the ones that have better support for these features. There's more information about what is supported and what is not on this other third-party library's site: http://recordrtc.org/

So, the support for audio+video recording will be pursued in the future when browsers and devices have developed better support for these features. ICE-11184 was created to this end.

Tested with ICEfaces 4 trunk r49573, Tomcat 7.

Android 6.0.1 testing
Non SSL camera - Only able to toggle with native controls.

SSL camera - A non-native toggle camera button is now shown. When using the toggle camera button the viewport for the picture is too large to fully fit on screen, because of this there are no camera controls available. PLease see attached screen shot toggledView.png.
Also in SSL mode the camera does not switch to video mode so that the user can change the shot. It stays in the very first frame it focuses on and can not be changed.

Spinner is now properly removed after uploading the file.

When testing camera on the android I do not get an option to install bridgeIt at any time. Not sure if this is an issue or not.

ios 10.0.2 testing
SSL and non-SSL Camera - A subsequent photo can now be taken with the camera demo, however with each attempt the user still sees the nag screens (Safari can not open the page because the address is invalid" and "Having Problems? BridgeIt may not be installed" dialog). The nag screens must be dismissed each time before taking a photo.

Android and ios SSL and non-SSL - Image dimensions attribute fails to resize the image. This was not an issue when using bridgeIt and ICEfaces 4 trunk Jenkins build 2172 prior to the changes made in this JIRA.

IE 11 - After installing the latest updates I am able to use the camera demo without issues.

r49708: removed facingMode attribute from mobi:camera; applied logic to cycle through available cameras on the system by specifying their device IDs

r49710: mobi:camera, fix to jump directly to using the native file upload input on iOS devices and thus avoid the bridget-related nag screens

ICEfaces 4 trunk r49711.
Verified camera jump directly to native file upload using ios 10.2 ipad.

r49716: mobi:camera, fixed various issues on Android: fixed width and height issues of camera popup and fixed overlay positioning and behaviour to initiate video stream; removed unnecessary code related to camera toggling; added mobile OS detection

Regarding the Android issues above:

Non SSL camera - Only able to toggle with native controls.

This is expected. Chrome has the security policy of only allowing video an audio stream for SSL enabled servers.

SSL camera - A non-native toggle camera button is now shown. When using the toggle camera button the viewport for the picture is too large to fully fit on screen, because of this there are no camera controls available. PLease see attached screen shot toggledView.png.

These issues were fixed with this commit.

Also in SSL mode the camera does not switch to video mode so that the user can change the shot. It stays in the very first frame it focuses on and can not be changed.

The overlay to initiate the video stream on Android was fixed with this commit too, so it's shown and positioned properly. Android only allows camera and audio streams from direct user actions, so in this case, the video stream will start when touching this overlay.

When testing camera on the android I do not get an option to install bridgeIt at any time. Not sure if this is an issue or not.

This is expected if the HTML5 camera is used. We do not display the Bridget messages if the HTML5 camera is launched. We only do so when we fall back to the native file upload input, except on iOS.

Android and ios SSL and non-SSL - Image dimensions attribute fails to resize the image. This was not an issue when using bridgeIt and ICEfaces 4 trunk Jenkins build 2172 prior to the changes made in this JIRA.

maxwidth and maxheight seem to be Bridgeit-only features. In the camera js code, I see those parameters only sent to the Bridgeit call.

Re-opened to attempt to implement support for the maxHeight, maxWidth attributes in the HTML5 camera.

ICEfaces 4 trunk r49716

android 6.0.1 nexus 5 phone tested on Labs 1 SSL server.
The issue of the toggled view being too large for the screen is resolved, however toggled view still fails. No camera image is shown in the viewport after toggling to rear camera. See attached screenshot toggledView2.PNG.

When using the front camera in portrait mode, the preview image and the uploaded image are stretched horizontally by about 40%. When using landscape mode this is not an issue.

Testing with r49716/ on MsEdge38 and FF49, when trying to use the Camera while using image button, take picture, then Redo, then Cancel, the image path is rendered on the demo as in attached screen shot (Camera.PNG).

r49721: mobi:camera, changed getUserMedia API settings for Android devices to try to make multiple cameras work on the same page

Please test to see if the rear camera now shows its video stream.

r49725: mobi:camera, fixes to be able to see video streams from both cameras when toggling them on Android devices

Please try this new fix. It worked on my Android device.

ICEfaces 4 trunk r 49725, local SSL tomcat 7, android 6.0.1 nexus 5 phone
The rear facing camera now shows the video screen. After taking a picture or pressing cancel, the opposite camera no longer has a video stream. Once this occurs, pressing the Next Camera button repeatedly causes both camera to fail to render video stream.

r49726: mobi:camera, fix to stop open video streams when re-launching the camera and when taking a picture in order to allow a new camera stream to display

ICEfaces 4 trunk r49726, FF 47, Chrome 53, MS Edge
Issue found with desktop browsers when using the redo button. The viewport collapses. See attached screenshot collapse.PNG. After pressing Take Picture again the viewport is restored but is rendered empty. A console error is seen:
TypeError: video.parentElement is null
http://localhost:8080/showcase/javax.faces.resource/coalesced.js.jsf?ln=ice.core&dgst=xe4dyo line 4268 > eval
Line 386

r49732: mobi:camera, fix for disappearing video frame after clicking redo button

This fixes the issue above.

ICEfaces 4 trunk r49732, local Tomcat 7 SSL server, FF 47, Chrome 53, MS Edge 38, android 6.0.1, ios 10.2
The disappearing video frame image issue is now resolved. Since this JIRA is getting long I will summarise the remaining issues to this point:

• When taking a photo using android front or rear facing camera in portrait orientation, the preview and uploaded photos are streched horizonatally
• When taking a photo using MS Edge the preview, thumbnail and uploaded images are skewed.
• With android device, repeatedly toggling cameras without taking photo or launching video stream results in continual overlays darkening the viewport. After 10 or so toggles the viewport is black.
• When trying to use the Camera while using image button, take picture, then Redo, then Cancel, the image path is rendered on the demo as in attached screen shot (Camera.PNG).
• Image dimension settings are not respected.

ICE-11176 verified disappearing video frame, ICEfaces 4.2.0.BETA build 2 on Chrome 53, FF 47, ios ipad 10.2, android 6.0.1
Awaiting verification with MS Edge.
Issues in above comment are known issues for BETA release.

r49741: fix to avoid setting the configured height of the picture and, instead, using the height of the video element, based on the configured width, letting the browser determine the height, in order to cover different aspect ratios used by different browsers and devices; added logic to produce the thumbnail image proportionally to the photo dimensions

This commit fixes the stretching issues described in the first two points above.

ICEfaces 4.2.0.BETA Jenkins build 3. Tomcat 7 SSL, ios 10.2 ipad, android 6.0.1, FF 47, Chrome 53
Verified the horizontal image stretching issue on android camera resolved. Awaiting verification with MS edge.

Two additional issues were noticed:
1.) On android device if changing the orientation of the device from portrait to landscape while the "Touch to start camera screen" modal layer is active, the layer does not resize to the landscape viewport. See attached screenshot modal.png
2.) On the ipad, after taking a front facing or rear facing photo in portrait mode, the thumbnail image is rotated 90 degrees counter clockwise. When taking a front facing photo in landscape mode the thumbnail is rendered upside down. When taking a rear facing photo in landscape mode the thumbnail is rendered in the proper orientation.
In all cases the uploaded image is rendered in the correct orientation.

Just to mention that those last two issues above weren't caused by the last commit. I had noticed the need for those improvements. They shouldn't affect the beta release.

Agreed, just added to the known issues list for record keeping.

r49785: fixes for Android to remove the overlay when changing cameras and to re-place it when changing the device's orientation; fix to correctly update the button markup when pressing the 'redo' button with either the original button image or the label with the original styling

r49790: added support for the maxwidth and maxheight attributes on the HTML5 camera

Testing Notes: this feature only works on platforms that support image cropping/resizing. This feature never worked on Edge in all the tests I made. Also, desktop Chrome and Firefox always added extra blank (transparent) space to the images (right and bottom) when maxwidth or maxheight were specified (about 5% of the magnitude). More accurately, the captured image is being sized at 95% of the specified dimensions and the rest is transparent blank space.

r49797: fixed thumbnail orientation on iOS devices

r49799: removed script that removed maxwidth and maxheight controls for desktop clients in showcase demo

r49811: fix to avoid 'This type of audio file isn't supported' message in audio played on Edge when starting recording

ICEfaces 4 trunk r49811, Tomcat 8, android 6.0.1 nexus 5 phone. HTML 5 camera.
Desktop browsers FF 47, Chrome 55
Initial test of image dimension settings shows that the image in the thumbnail and the uploaded file are sized based on the length and with from the top left corner. Please see attached screenshots uploaded.png and thumbnail.png. Before taking this photo the image dimensions were set to 50 x 50. While aiming the camera the viewfinder showed my whole face. After taking the photo, the thumbnail and uploaded image are cropped to 50 x 50 starting at the top left corner, instead of scaling the whole image to 50 x 50. This results in only the upper left of the face captured in the image.
Issue seen with android Chrome as well as desktop FF and Chrome.

When testing HTML 5 camera on ipad ios 10.2 the image dimension settings do not function in the demo.

I've tried to reproduce the issue above on desktop Chrome, desktop FF, and Android (Nexus 7 tablet), and the image resizing is working well for me in all cases. When setting the maxwidth and maxheight settings to 50 each, the resulting image is resized, not cropped to those dimensions. See attached screenshots AndroidCamera[1,2,3].png. The picture is of my laptop, and it appears completely in the resulting image.

Please report if the issue seen is reproducible with a specific series of steps.

The image size settings just won't work on iOS, because of it's lack of support of the getUserMedia API. On iOS we can only use the native camera, and we can't specify maximum width and maximum height for the resulting image.

Tested with ICEfaces 4 trunk continuous Jenkins build 2306, Tomcat 8.
I am no longer able to reproduce the cropping issue in Chrome or Firefox desktop browsers or on android 6.0.1.
The image dimensions settings appear to have stopped functioning on the android device. Images are no longer resized.

r50009: modified mobi:camera demo for easier, more intuitive usage and testing
r50010: paraphrased TLD documentation for maxwidth and maxheight attributes for easier understanding

The image dimension settings are working well for me on Android and desktop Chrome and FF. Perhaps you sometimes missed pressing the "Set Dimensions" button to submit the values to the server. I modified the demo to use ajax when changing any of the max width or max height fields to update the entire page, in order to make sure that the new values are applied to the mobi:camera component, which is in a different form.

The comment I made Jan 11 was in error as I was not testing the camera app on an SSL server as I should have been.
When testing the camera demo in SSL mode, the thumbnail and uploaded image still appears to be cropped to a distance from the top left hand corner. This issue was found on 2 android nexus 5 devices as well an an android S6 edge phone.

When testing the camera demo in NON-SSL mode the image dimensions do not function on the android devices

Couple of new regressions, likely due to changes I made for ICE-11226. (sorry).

On Safari and IE11 (presumably any browser that doesn't support .getUserMedia apis), nothing happens now when you press the camera or microphone button.

Users should see a "Sorry the camera is not supported on this platform." dialog, After dismissing the dialog you can browse for an image from the local filesystem instead.

Also note using svn rvn#50524 on Firefox 51.0.1 on MacOS, the viewfinder doesn't render, and taking a picture doesn't work (seems to be empty?).

r50526: re-enabled fallback file upload controls when HTML5 and/or the getUserMedia API aren't supported in the browser

The dialog is not shown yet. Should this be a native dialog or some dialog similar to the previously shown by bridgeit?
Please test if the FF on MacOS issue is fixed as well.

Art to implement "Access to the camera is not supported in this browser." dialog.
Also, add file upload fallback feature to the mobi:microphone.

r50539: added file upload fallback feature to mobi:microphone; added "Access to the camera is not supported in this browser." dialog and the corresponding dialog for the microphone

ICEfaces4 trunk r 50539/ desktop Chrome55, MsEdge38, IE11; iOS10 Safari/iPad testing:

Safari, iOS10, iPad
Camera/Microphone/ SSL and nonSSL
The message, "Access to the camera is not supported in this browser. Falling back to file upload." appears always on both demos, on top of the file picker button.

Another issue, on the microphone demo (SSL and nonSSL), clicking the file picker button opens up dialog to open Camera and take photos. This is not an issue in desktop browsers.
This was previously reported by Liana in this comment: http://jira.icesoft.org/browse/ICE-11176?focusedCommentId=64537&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-64537

Desktop/all browsers/nonSSL:
The message "No camera is available or there was an unknown error. Falling back to file upload." appears on both Camera and Microphone demos on top of the "Choose file" button.

Android:
There are no changes or regressions in Android functionality.

Seeing it in action, I think we should remove the popup message altogether. Just render the browse button in cases where the camera is not directly accessible.

Also, can the microphone upload on iOS access the microphone, and sound files to upload, instead of trying to use pictures/videos?

r50549: removed "Access to the camera/microphone is not supported in this browser" dialogs; disabled mobi:microphone button on iOS since the file upload input cannot be configured to select audio files

It's not possible to make the file upload input select audio files on iOS. We are using the correct markup to do so, but iOS simply doesn't support it. This can be verified on the following test page https://mobilehtml5.org/ts/?id=23

1.) The thumbnail and uploaded photos are not properly resized when using image dimensions setting. Instead they are cropped to the top left corner .

This is considered a Known Issue, since it doesn't happen on all Android devices.

2.) When recording with the microphone the time keeper does not advance, it stays at zero. However the recording and upload are successful.

It's not supposed to, what you are seeing is the player that is used to preview the recording once you've captured it. It probably shouldn't appear until the recording is captured, to avoid confusion.

Microphone demo - the "Record Audio" button was disabled on iOS, however the "Upload Recorded Audio" button is enabled, and if pressed, the message is rendered: "The uploaded audio file could not be correctly processed."

If the demo can generically be made to only enable the Upload button when a captured image/audio clip is available, then let's do that. Otherwise, we'll consider this a Known Issue.

r50555: added script to disable upload button on iOS (on the mobi:microphone showcase demo)

The component doesn't have any callbacks or listeners for the moment when audio is captured, so it's not possible to easily only enable the upload button when the audio has been captured.

r50567: mobi:microphone, added timer and red dot recording indicator; fixed audio player size when changing orientation to fit in the available width; fix to only make audio player visible after the recording has been made

ICEfaces 4 trunk r50730
android 6.0.1
These issues are now resolved.
"2) When recording with the microphone the time keeper does not advance, it stays at zero."
"3) When recording a sound file in portrait mode the timer part of the recording area extends outside the boundaries of the dialog. "